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ON THE INFORMATIONAL STRUCTURE IN OPTIMAL DYNAMIC 

STOCHASTIC CONTROL 


SAUL JACKA AND MATIJA VIDMAR 

Abstract. We formulate a very general framework for optimal dynamic stochastic control prob¬ 
lems which allows for a control-dependent informational structure. The issue of informational 
consistency is investigated. Bellman’s principle is formulated and proved. In a series of related 
results, we expound on the informational structure in the context of (completed) natural filtrations 
of stochastic processes. 


1. Introduction, a convention and some general notation 

1.1. Introduction. Optimal dynamic stochastic control, is stochastic, in the sense that the output 
of the system is random; optimal in that the goal is optimization of some expectation; and it is 
dynamic, in that the control is adapted to the current and past state of the system. In general, 
however, the controller in a dynamic stochastic control problem can observe only some of the 
information which is being accumulated; and what is observed may vary, depending on the chosen 
control. 

It is reasonable, therefore, to insist that all admissible controls (when regarded as processes) are 
adapted (or even predictable with respect) to a control-dependent-information-flow that is being 
acquired by the controller. 

Some (informal) examples follow. 

Example 1.1. Job hunting. Consider an employee trying to optimize her choice of employer. 
The decision of whether or not to move, and where to move, will be based (in particular) on the 
knowledge of the qualities of the various employers that she can choose from. This knowledge itself 
will depend on her previous employers and the dates she spent there. Costs are associated with 
moving, and resources can be devoted to determining the quality of employers before a move. 

Example 1.2. Quality control in a widget production line. A fault with the line may cause all the 
widgets from some time onwards to be faulty. Once this has been detected, the line can be fixed and 
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the production of functioning widgets restored. However, only the condition of those widgets which 
are taken out of the production line, and tested, can actually be observed. A cost is associated with 
this process of testing. Conversely, rewards accrue from producing functioning widgets. 

Example 1.3. Tanaka’s SDE. From the theory of controlled SDEs, a classic example of loss of 
information is Tanaka’s SDE: let X be a Brownian motion and Wt = fy sgn(X s )dX s , t 6 [0, oo). 
Then the completed natural filtration of W is strictly included in the completed natural filtration of 
X Ha p. 302], 

Example 1.4. Bandit models. The class of bandit models is well-known (see the excellent litera¬ 
ture overview in EEjj. These are “sequential decision problems where, at each stage, a resource like 
time, effort or money has to be allocated strategically between several options, referred to as the 
arms of the bandit ... The key idea in this class of models is that agents face a trade-off between 
experimentation (gathering information on the returns to each arm) and exploitation (choosing the 
arm with the highest expected value).” JEJ p. 2], 

Other situations in which the control is non-trivially, and naturally, adapted or predictable with 
respect to an information-flow, which it both influences and helps generate, abound; for example, 
controlling the movement of a probe in a stochastic field, only the local values of the field being 
observable (see Example |3.15[ ); a similar situation for movement on a random graph, the controller 
only being able to observe the values attached to the vertices visited; additively controlling a 
process, but observing the sum of the process itself and of the control. 

Analysis of dynamic stochastic control problems is vastly facilitated by the application of Bell¬ 
man’s principle. Informally this states that (i) the best control is to behave optimally now, con¬ 
ditionally on behaving optimally in the future, and that (ii) the longer one pursues a fixed, but 
arbitrary, policy before behaving optimally the worse one’s payoff from the control problem. 

Failure to reflect the control-dependent informational flow explicitly in the filtration structure, 
may result in Bellman’s super(martingale) principle not being valid. This is exemplified in the 
following: 

Example 1.5. Box picking. Let Ll c = {0,1} X {—1,1}, endow it with the discrete a-field H c = 2^ 
and the probability measure P, given by P({(0, —1)}) = 1/6, P({(0,1)}) = 1/3, P({(1, —1)}) = 1/3, 
P({(1,1)}) = 1/6. Let Y\ and \\ be the projections onto the first and second coordinate, respectively; 

def 

set Xq = 0. The time set is {0,1,2}. Note that Y\ stochastically dominates Yi and that on 
{Yi = 1}, {>2 < Y \}, whilst conditionally on {Y\ = 0}, Y^ has a strictly positive conditional 
expectation. 

The set of admissible controls are processes c = (cj)/ =0 such that cq is 0, while ci and C 2 are 
{l, 2}-valued with c predictable with respect to the natural filtration of the controlled process X c 

def 

which is given by Xf = Y Ct for t 6 {0,1,2}. We can think of being allowed to open sequentially 
either of two boxes containing rewards. 
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So, we decide on index 1 or 2 and then after observing the corresponding Y\ or Y^, decide again 

def 

on 1 or 2. The objective functional, whose expectation is to be maximized, is J(c ) = Xf + Xf for 
admissible controls c. 

For an admissible control c and t E {0,1,2}, let D(c,t ) be the set of admissible controls that 
agree with c up to, and inclusive of, time t. 

It is then elementary to verify that: 

(1) The control c* given by Cg = 0, c\ = 1 , c \ = 21{x 1= o} + 1 {a'i=i}> unique optimal 

control. 

(2) Setting Fq = f {0,12} and F\ = f F 2 = f 2 n , F = (Ft)t =0 is the smallest filtration with respect 
to which all the X c (as c runs over the admissible controls) are adapted. The “classical Bellman 
process” W* = (Wf )'( =Q associated to this filtration, i.e. the process 

W t * A == sup E p [J(d)\Ft\ for t E {0,1, 2}, 

d£D(c* ,t) 

is not an F-supermartingale-contradicting both parts of Bellman’s principle. 

(3) Letting Q c be the natural filtration of X c , the “informationally-consistent Bellman 
process” V c = (V} c )jL 0 associated to this filtration, i.e. the process 

V t c = sup E p [ J{d) I gf] for t€ {0,1, 2}, 

d£D(c,t) 


is a Q c -supermartingale for each admissible control c-confirming that this version satisfies the 
second part of Bellman’s principle; and V c is a Q c -martingale-confirming that this version 
satisfies the first part of Bellman’s principle. 


Item (3) also follows from Theorems 5.f and 5.1 below. 


So, one should like a general framework for stochastic control, equipped with a suitable abstract 
version of Bellman’s principle, which makes the control-dependent informational flows explicit and 
inherent in its machinery. The circularity of the requirement that the controls are adapted to an 
informational structure which they themselves help generate, makes this a delicate point. It would 
seem that this is one aspect of general dynamic stochastic control which has not yet received due 
attention in the existing literature. Hitherto, only a single, non-control dependent, observable [5] 
informational flow mm appears to have been allowed. This is not to say that the phenomenon has 
not entered and been studied in the literature in specific situations/problems, see e.g. [6J ;8] El [19] 
[U Chapter 8] and the references therein; the focus there having been (for the most part) on 
reducing the original control problem, which is based on partial control-dependent observation, to 
an associated ‘ separated ’ problem, which is based on complete observation. The exception is the 
monograph of Yiiksel and Ba§ar [20] . It provides a framework for stochastic optimization under 
information constraints with (possibly) multiple agents and non-perfect recall. It does not, however, 
offer an explicit general version of Bellman’s (super)martingale principle. 
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In this paper we attempt to fill this gap; describing a general (single controller, perfect recall) 
stochastic control framework, which explicitly allows for a control-dependent informational flow, 


and providing (under a technical condition, Assumption 5.3), a fully general, abstract version of 


Bellman’s principle. This is the content of sections 3-6. Specifically, section [3] formally defines 
a system of stochastic control (in which observed information is an explicit function of control); 
section [4] discusses its conditional payoff and ‘Bellman’ system; section [5] formulates the relevant 


version of Bellman’s principle - Theorem 5.7 is our main result. We should emphasise that for this 
part of the paper we follow on from the approach of El Karoui f5j. 

Section [6] contains a detailed solution of a concrete class of examples in some reasonably straight¬ 
forward cases, illustrating some of the main ideas of this paper. Several other examples and coun¬ 
terexamples are also given along the way. 

A crucial requirement for the programme outlined above to be successful is that of informational 


consistency over controls (see Definition 3.12): if two controls agree up to a certain time, then what 
we have observed up to that time should also agree. Especially at the level of stopping times, this 
becomes a non-trivial statement - for example, when the observed information is that generated 
by a controlled process, which is often the case. We explore this issue of informational consistency 
in the context of (completed) natural filtrations of processes in section [7| Specifically, we consider 
there the following question, which is interesting in its own right: 

Question 1.6. Suppose that X and Y are two processes, that S is a stopping time of one or both 
of their (possibly completed) natural filtrations; and that the stopped processes X s and Y s agree 
(possibly only with probability one): must the two (completed) natural filtrations also agree at the 
time 57 

The answer to this question is non-trivial in a continuous-time setting, and several related findings 
are obtained along the way (see the introductory remarks to section [7j on p. 26, for a more detailed 
account). In essence they are connected with a generalization (Theorem |7.9[ ) of a part of Galmarino’s 
test, available for coordinate processes on canonical spaces [3, p. 149, Theorem IV. 100] [12; p. 320, 
Lemma 4.18], and extended here to a non-canonical setting. In particular, in existing work one 
finds argued, under reasonably innocuous conditions (e.g. m p. 9, Proposition 2.18]), that the 
(7-field generated by the stopped process is included in the history of the natural filtration of the 
process up to that stopping time. The opposite inclusion, has (to the best of our knowledge) until 
now only been established for coordinate processes on canonical spaces [TH] p. 33, Lemma 1.3.3] or, 
slightly more generally, under the condition that all the stopped paths of the process are already 


paths of the process [T5l . Paragraph 1.1.3, p. 10, Theorem 6]. We show in Theorem 7.9 that it holds 
true far more generally. 







ON THE INFORMATIONAL STRUCTURE IN OPTIMAL DYNAMIC STOCHASTIC CONTROL 


5 


1.2. A convention. As stated earlier, we will provide and analyze a framework for optimal dy¬ 
namic stochastic control in which information is explicitly control-dependent. The informational 
flow is modeled using hltrations, and this can be done in one of the following two, essentially 
different, ways: 

(1) Dealing with events ‘with certainty’, irrespective of the presence of probability. 

(2) Dealing with events ‘up to a.s. equality’, insisting that the hltrations be complete relative 
to the underlying probability measure(s). 

In sections [|-[5| we will develop the second ‘probabilistic’ approach - of complete hltrations - 
in parallel to the hrst - ‘measure-theoretic’ - setting. The formal differences between the two 
approaches are minor. For the most part one has merely to add, in the ‘complete’ setting, a 
number of a.s. qualifiers. We will put these, and any other eventual differences of the second 
approach as compared to the hrst, in {} braces. This will enforce a strict separation between the 
two settings, while allowing us to repeat ourselves as little as possible. 

1.3. Some general notation. Throughout this paper, for a probability measure P on Q and 
A C 12, a property will be said to hold P-a.s. on A, if the set of cu € A for which the property does 
not hold is hrst measurable (i.e. belongs to the domain of P), and second is of P-measure zero. 
When A = ft, we shall just say that the property holds P-a.s. 

For £ a collection of subsets of ft, 

(i) cr(£) = crfi(C) denotes the er-held generated by £ (on ft)-, 

Hpf 

(ii) if A C D, C\a — {L n A : L G £} denotes the trace of £ on A. 

(iii) Given a map / from ft into some measurable space (E,£), cr(f) = afi({f~ 1 (A) : A £ £}) 
is the c-held generated by / (the space (E,£) being understood from context); 

(iv) 2 X denotes the power set of a set X. 

2. TWO MOTIVATING EXAMPLES 

We start with two key examples which we shall revisit and formalise later. 

Example 2.1. Switching between two Brownian motions. We may observe each of two independent 
Brownian motions (BMs), B° and B 1 , but at any given time we may only observe one of them. We 
acquire a running reward of the difference between the BM we currently observe and the unobserved 
one. At any time we may pay a state-dependent cost K to switch our observation to the other BM. 
The control process, c, gives the index of the BM we choose to observe, and is any cadlag process 

def 

taking values in {0, 1}, predictable with respect to the filtration Q(: = (t(B(. s : s < t) (and satisfying 
certain restrictions on the time between jumps that we will specify later on p. [?|). 

We denote by a c (t) the last time that we changed our observed BM i.e. the last jump time of c 
before time t, and r c (t ) is the lag since the last jump i.e. r c (t) = t — cr c (t). Then we define Z c as 


ON THE INFORMATIONAL STRUCTURE IN OPTIMAL DYNAMIC STOCHASTIC CONTROL 


6 


follows: 


r/c def tjc nl-c 


so that Zf is the conditional mean of B} — B\ c given observations up to time t. 
The reward J (which we seek to maximise) is given by 


Ac) = / e 
Jo 


oo /*oo 

~ at Zfdt- / e~ at K(Z(_,T(t-))\dc\ t . 

Jo 


We formalise this example on p. [?| generalise it in Example 3.15 , and solve it in some special cases 
in section 0 


Example 2.2. Poisson random measure search model. The controlled process X is a process in 
M n . The control c is the drift of X and is bounded in norm by 1 so 

X t c = [ c s ds + aW t , 

Jo 

where W is a BM and ||cj|| < 1. 

There is an underlying locally finite Poisson random measure on M n , /i. 

We (progressively) observe W and the restriction of p, to the path traced out by B(X C ), the closed 
unit ball around X c (a drift-controlled Wiener sausage mi 
The objective function J (to be minimised) is 


J{c)= e 

Jo 


—at 


A B ( X t)) dt + e “ TC « 1 (r<=<oo)i 


where t c is a stopping time (time of retirement) which we also control and k is the cost of retirement. 


This example is formalized and generalized as Example 3.16. 


3. Stochastic control systems 

We begin by specifying the formal ingredients of a system of optimal dynamic stochastic control. 

Definition 3.1 (Stochastic control system). A stochastic control system consists of: 

(i) A (linearly ordered) time set T. 

We will assume (for simplicity) that either T = No, or T = [0, oo), with the usual order. 

(ii) A set C of admissible controls. 

For example, {equivalence classes of} processes or stopping times. In general, C is an 
arbitrary index set. 

(iii) A sample space fl endowed with a collection of u-algebras (J 7C ) cg c- 

We regard J- c as all the information accumulated (but not necessarily acquired by the 
controller) by the “end of time” or, possibly, by a “terminal time”, when c is the chosen 
control. For example, in the case of optimal stopping, given a process X, the set of controls 
C would be the {equivalence classes of the} stopping times of the {completed} natural 
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filtration of X, and for any S G C, F s = o(X s ), the o-field generated by the stopped 
process {or its completion}. 

(iv) (P c ) ceC , a collection of {complete} probability measures, each P c having a domain which 
includes the {P c -complete} u-field F c (for c G C). 

The controller chooses a probability measure from the collection (P c ) c eC- This allows for 
the Girsanov approach to control, where the controller is seen as affecting the probability 
measure, rather than just the random payoff. 

(v) A reward function J : C — > [— 00 , 00 ] , each J(c) being ^-measurable {and defined up to 
P c -a.s. equality} (as c runs over C). 

We further insist E p J(c)~ < 00 for all c G C. Given the control c 6 C, J(c) is the 
random payoff. Hence, in general, we allow both the payoff, as well as the probability law, 
to vary. 

(vi) A collection of filtrations, indexed by T, (G c ) c & c on Q. 

Hpf 

It is assumed G%o — VterGt C F c , and (for simplicity) that Go is P c -trivial (for all c £ C) 
{and contains all the P c -null sets}, while Gq = Go { so the nu U sets for P c and P d are the 
same} and P c |gg = P rf | gd for all {c, d} C C. Gf is the information acquired by the controller 
by time t G T, if the control chosen is c G C (e.g. G c may be the {completed} natural 
filtration of an observable process X c which depends on c). Perfect recall is thus assumed. 


Example 2.1 continued. We formalize Example \ 2. 1\ in the context of Definition 3.1 The time 
set is [0, 00 ). Fix the discount factor a G (0, 00 ), let (Gl,T-L, P) be a probability space supporting two 
independent, sample-path-continuous, Brownian motions B° = (-B^e^oo) and B 1 = (Bl) te r 0)OO ), 
starting at 0 and —x G M, respectively (Definition sm and |3.fivj[ - T c = H, P c = P for all c). 
Denote by T the natural filtration of the pair (B°, B 1 ). Then, for each cadlag, IF-adapted, {0,1}- 
valued process c, let G c be the natural filtration of B c , the observed process (Definition 
let (J{.)% 1 0 he the jump times of c (with J(j = f 0; ./{' = 00 , if c has less than k jumps); and define: 


C '= [J {F-adapted, cadlag, {0, l}-valued, processes c, with cq = 0, 


(3.1) 


e>0 


that are G c -predictable and such that J{] +1 — J{. > e on {Jjj < 00 } for all k G No} 
(Definition 3.1(H)). A more “obvious” choice would be to omit the condition on ‘e-separation 
of jumps’ of the control process in the definition of C; however, a moment’s thought will show 
that an optimal control is potentially not cadlag, since, on jumping, we are exposed to unpleasant 
surprises: the unobserved BM may be so highly negative that we wish to switch straight back 
on first observing it. The insistence on the “e-separation” of the jumps of the controls allows 
us to emphasize the salient features of the control-dependent, informational flow, without being 
preoccupied by the technical details. 

Next, define, for each c G C; 

• the last jump time of c before time t and the lag since then, respectively: 
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- of = sup{s E [0 ,t] : c s c t }; 


c def c 

Tt =t- (J c t ; 


for each t E [0, oo) (with the convention sup0 c = 0); 

Z c = f B c — B)ff c , the current distance of the observed Brownian motion to the last recorded 
value of the unobserved Brownian motion; 

J(c ) l = f / 0 °° e~ at Zfdt—^ e~ at K(Zf_,Tf_)\dc\t, where K : Mx [0, oo) —> M is a measurable 
function of polynomial growth (Definition 3.1 (v)' note that \Z C \ < B° + B 1 (where a line 


over a process denotes its running supremum), so there are no integrability issues (thanks 
to the ‘e-separation’ of the jumps of c). 


Notice that E p [/“ e~ at Zfdt] = E p [/“ e~ at ( B c t - B]~ c ) dt\. Define V(x) d = sup ceC E p J(c). o 
Definition 3.2 (Optimal expected payoff). We define the optimal expected payoff 

def rP c t / \ 

v = sup E J (c) 
cSC 

(with sup0 = —oo). Then, c E C is said to be optimal if E pc J(c) = v, while a C-valued net is said 
to be optimizing if the limit of its payoffs is v. 

Remark 3.3. 

(1) It is no restriction to assume the integrability of the negative part of J in Definition 3. M 
since, allowing controls c for which E pc J(c) = oo, but for which E pc J(c) is defined, would 
not change the value of v. 

(2) A consideration of filtering problems shows that it is not natural to insist on each J(c) 
being (/(^-measurable. The outcome of our controlled experiment need never be known to 
the controller, all we are concerned with is the maximization of its expectation. 

(3) In the case where C is a collection of processes, a natural requirement is for each such 
process c E C to be adapted or previsible with respect to Q c . If C is a collection of random 
times, then each such c E C should presumably be a (possibly predictable) stopping time 
of Q c . But we do not insist on this. 


We now introduce the concept of a control time, a natural generalization of the notion of a 
stopping time to the setting of control-dependent filtrations. 

Definition 3.4 (Control times). A collection of random times S = (5 c ) ce c is called a control 
time, if S c is a {defined up to P c -a.s. equality} stopping time of Q c for every c E C. 

Example 3.5. A typical situation to have in mind is the following. We observe a process X c , its 
values being dependent on c. Then define Q c to be the {completed} natural filtration of X c . Letting 
S c be the first entrance time of X c into some fixed set, the collection (5 c ) ce c constitutes a control 
time (as long as one can formally establish the stopping time property). o 
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Definition 3.6 (Deterministic and control-constant times). If S c (uj) = a G T U { 00 } for {P c - 
almost} all ui G fl, and every c G C, then S is called a deterministic time. More generally, if 
there is a random time S, which is a stopping time of Q c and S c = S {P c -a.s} for each c G C, then 
S is called a control-constant time. 


As yet, C is an entirely abstract set with no dynamic structure attached to it. The following def¬ 
initions establish this structure. The reader should think of T>(c, S) as being the controls “agreeing 
{a.s.} with c up to time 5” (see Examples 3.15 and 3.16 for examples of the collections T>(c,S)). 


Definition 3.7 (Adaptive control dynamics). Given a stochastic control system of Definition 3.1 


the system is said to have adaptive control dynamics if it comes equipped with a family 
(D(c, 5))( C i 5 ) eCxG of subsets of C with the following properties: 


(1) G is a collection of control times. 

(2) c G V(c,S ) for all (c, S) G C x G. 

(3) For all S G G and {c, d} C C, d G V(c,S) implies S c = S d {P c and P d -a.s}. 

(4) If {S, T} C G, c G C and S c = T c {P c -a.s}, then V(c, S ) = V(c, T). 

(5) If {5,T} C G and c G C for which S d < T d {P d -a.s.} for d G V(c,T), then V(c,T) C 

V(c,S). 

(6) For each S G G, {D(c, S) : c G C} is a partition of C. 

(7) For all (c, S) G C x G, if S c is identically {or P c -a.s.} equal to 00 (respectively 0) then 

T>(c,S ) = {c} (respectively V(c,S ) = C). 


Remark 3.8. In connection with Condition | (7) | of Definition 3.7 see Remark 5.1 [iii)| 

Definition 3.9. When condition | (2) | and |(6)| of Definition 3.7 prevail, we write for the equiva¬ 
lence relation induced by the partition {P(c, S) : c G C}. 


Remark 3.10. When S is not a control-constant time then condition (3) of Definition 3.7 may 
bite. For example, when, for some control c, S c is the first entrance time into some fixed set of an 
observed controlled process X c , then controls agreeing with c up to S c , should leave X c invariant. 
This assumption is thus as much a restriction/consistency requirement on the family T>, as on which 
control times we can put into the collection G. Put differently, G is not necessarily a completely 
arbitrary collection of control times. For while a control time is just any family of t/ c -stopping 
times, as c runs over the control set C, the members of G enjoy the further property of “agreeing 
between two controls, if the latter coincide prior to them”. This is of course trivially satisfied for 
deterministic times (and, more generally, control-constant stopping times), but may hold for other 
control times as well. The choice of the family G will generally be dictated by the problem at hand, 
and is typically to do with what times the act of “controlling” can be effected at. Example |3.11 


below, anticipating somewhat the results of section [5j illustrates this in the control-independent 
informational setting. It manifests, if one is to preserve Bellman’s principle, the need to work with 
stopping times (and a fortiori control times, when there is dependence on control) in general. 
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Example 3.11. Control at stopping times. Fix a probability space (17, TL, P) and on it (i) a Poisson 

def 

process TV of unit intensity with arrival times (S n ) n ^ 0 , So — 0, S n < oo for all n £ N; (ii) an 
independent sequence of independent random signs R = (R n )n&n 0 with values in {—1,+1} with 
P (Rn = +1) = 1 - P (Rn = -1) = 2/3. 

The “observed process” is 

W t = N t + f Y. Rnl [Sn ,S n+1 )(s)ds 

Jo neN 0 


(so we add to TV a drift of R n during the random time interval [S n ,S n+ i), n> 0). Let Q be the 
natural filtration ofW. Remark the arrival times of N are stopping times of Q. 

The set of controls C consists of real-valued, measurable processes, starting at 0, which are 
adapted to the natural filtration of the bivariate process (Wl^A^o}! N) (where AN is the jump 
process of N; intuitively, we must decide on the strategy for the whole of [S n ,S n+ i) based on the 
information available at time S n already, n > 0). For X £ C consider the penalty functional 

J{X) / e _a *l (0 ,oo) o \X t - W t \dt, 

J [0,oo) 


where a £ (0,oo). Let v = infxec E P J(X) be the optimal expected penalty; clearly an optimal 
control is the process X which takes the value of W at the instances which are the arrival times of 
N and assumes a drift of+1 in between those instances, so that v = l/(3a). Next, for X £ C, let 


Vs = P-essinf yeC)yS=xS E p [J(F)|£s], 


S a stopping time of Q, 


be the Bellman system. We shall say Y £ C is conditionally admissible at time S for the control 
X, ifY s = X s and conditionally optimal at time S if, in addition, V(f = E p [J(Y)|£/s] P-a.s. Set 
V = f V* for short. 

We now make the following two claims. 

Claim, 1 The process (Vt) tg [ 0iOO ) (the Bellman process (i.e. Bellman system at the deterministic 
times) for the optimal control), is not mean non-decreasing (in particular, is not a submartingale, 
let alone a martingale with respect to Q) and admits no a.s. right-continuous version; moreover, 
E p Vt < E p Vo for each t £ (0,oo). 

Proof of Claim 1. First, clearly Vq = v. Second, for t £ (0,oo), the following control, denoted 
X*, is, apart from X, also conditionally admissible at time t for X: It assumes the value of W at 
the instances of the arrival times of TV. and a drift of +1 in between those intervals, until before 
(inclusive of) time t ; strictly after time t and until strictly before the first arrival time of TV which is 
> t, denoted St, it takes the values of the process which starts at the value of W at the last arrival 
time of TV strictly before t and a drift of —1 thereafter; and after (and inclusive of) the instance St., 
it resumes to assume the values of W at the arrival times of TV and a drift of +1 in between those 
times. Notice also that RtA(t is not an arrival time of TV) £ Q t , where Rt = X^neN 0 ^n^[S„,S„ + i)(*)) 
i.e. Rtt(t is not an arrival time of TV) is the drift at time t, on the (almost certain) event that t is 
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not an arrival time of N, zero otherwise. It follows that, since X is conditionally admissible for X 
at time t: 

v t < E p [j(x)\g t ], 

so E p Vjl{R t =-t-i} < E p J(A^)l| Rt=+1 }.; whereas since X* is also conditionally admissible at time t 
for X: 

v t < E p [j(x*)\g t ], 

so E p ^l{R t =-i } < E p J(X*)l {iJt= _ 1} = E p J(X)l {Rt= _ 1} - E p J m) e-<«dtl {Rt= - 1} = 
E p J(A)l{ Rt= _ 1 | — ^e~ at (l — (by the Markov property of N and the independence of R and 

N). Summing the two inequalities we obtain 

1 


E p Vt <v- 


_—Oit 


3(1 +a) 

implying the desired conclusion (for the nonexistence of a right continuous version, assume the 
converse, reach a contradiction via uniform integrability). □ 

Claim 2 The process (b^^)neNo *+ however, a discrete-time submartingale (and martingale with 
X = X) with respect to (Gs n )n£N 0 > f or X E C. 

Proof of Claim 2. For X = X, the claim follows at once from the observation that X is 
conditionally optimal at each of the arrival instances of N. The submartingale property is shown 
in section [5j on p. 22 once Bellman’s principle (Theorem |5.7[) has been established. 


□ 


Using the dynamical structure of Definition 3.7, a key condition — is as follows: 


Definition 3.12 (Stability under stopping). We say that a stochastic control with adaptive dy¬ 
namics system is stable under stopping if for all {c, d} C C and S E G satisfying c d, we 
have Q c s c = Q^ d and P c |gg c = P d | g^. 

From now on, up to and including section [5j we shall assume that the conditions of Definitions 


|3.1 3.7 and 3.12 all hold i.e. we are dealing with a stochastic control system which has adaptive 


dynamics and is stable under stopping which we shall refer to simply as a coherent control 
system. 

Remark 3.13. So, from now on it is appropriate to think of V(c, S ) as just V(c, S c ), the collection of 
admissible controls which: up to the stopping time S c , agree with c; generate the same information 
and have the same control law as c. 


Example 2.1 continued. For c G C and control times S, set 


V(c,S) = {de C : d^ = c^}. 


. uS c _ „S C 


Then let G be any subset of 


G' = f {control times S such that whenever d G T>(c,S) then S c = S d and Qg c = Gtd} 
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This defines a coherent control system. Clearly the deterministic times [0, oo) C G'. Moreover, 
thanks to Corollary 7.10 and Remark 7.7 G' = {control times S such that Vc E C Vd E C (d s ‘ = 


cr implies that S c = S d )}, provided is Blackwell. Note the freedom in the choice of G (as 


long as it is a subset of G'); see also Remark 3.1 4 (IV) to follow. 


We will conclude this section with rather general extensions of Examples |2.1| and 2.2 illustrating 
the concepts introduced thus far, focusing on the control-dependent informational flows, and with 
explicit references made to Definitions 3.1, 3.7 and 3.12| 

Before doing so, we summarize the remarks and observations regarding control systems made so 
far in 

Remark 3.14. Fundamental remark. 

(I) As far as the optimal value and optimal control (or optimizing net of controls) are concerned, 


(ii) 

(v) 

of Definition 

/Ct 

3.1 


satisfying items (ii) (v) of Definition 3.1, one can define T c = No or T = f [0, oo) , G = f 0 
and Gt = {0> D} for all c E C and t E T, and the system is a coherent control system, and 


also (trivially) satisfies Assumption 5.3 (to follow). Nevertheless, this is obviously a static 


description that ignores any dynamical structure. 

(II) Of course, the G c s (which require the presence of T) will typically be used to define C by 
insisting that an admissible control c be adapted/predictable/a stopping time with respect 
to G c (meaning, in particular, that the G c s should first be given a priori on a larger class of 
controls c, not just the ones that ultimately end up being admissible). But we do not insist 
on this, chiefly since it is not necessary to limit the controls to being processes or random 
times. 

(Ill) The rest of the stochastic control system structure is there to reflect a dynamic control 
setting and to facilitate its analysis. In particular, the equivalence classes, T>(c,S), specify 
a dynamical structure for the admissible controls c € C and those control times S, for 


which this can be done both consistently (see Definitions 3.7 and 3.12) and informatively 


(see Bellman’s principle, Theorem 5.7, below). 


(IV) Ceteris paribus, the bigger G, the more informative is Bellman’s principle. But there are 


restrictions on the members of G (especially those of Definition 3.12 and Assumption 5.3 


to follow), which will generally preclude some control times from being included in G. 


Specifically, Assumption 5.3 includes an interplay between (D, (J 7C ) ce c, (P c ) c eC> J, C) and 


(T, (G c ) C £ Ci G, (P(c, <S)) cG c,5gg)i the parts of the control system that, respectively, deter¬ 


mine and help analyze the stochastic problem at hand (see also Remark 5.5). In particular 


Assumption 5.3 can fail for deterministic times (see Example 3.11 as continued on p. 22) 


Remark 3.14 is most easily appreciated in the context of the two examples that follow. The first 
is a generalization of Example |2.1| to motion in a random field. 
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Example 3.15. Motion in a random field. 
We are given: 


The time set is [0,oo) (.Definition 3.1 (i): T = [0, oo)}. 


( 1 ) 

( 2 ) 


(3) 

(4) 

(5) 


a filtered probability space (fl, TL, J 7 , P) (Definitions \ 3. \ (Hi) and !T\(iv)\ T c = TL and P c = P 
for each c); 

• a subset O C M; 

• an “initial point” oo 6 O; 

• a random (time-dependent) real-valued field (Y°) 0& o - eachYfi being an Tt-measurable 
random variable, and the random map (( o,t ) i —> Y°) being assumed continuous from 
O x [0,oo) into M (so that the map (( ui,o,t ) i-a Y°{u)) is automatically TL <g> 13(0) <8> 
£>([0, oo))/ 13 (M.)-measurable) [in Example 2.1, O = {0,1}, oq = 0 and Y° = B° and 
Y l = B 1 are the two Brownian motions]; 

“discount factor a G [0, oo); 

a radius of observation a G [0, oo) [in Example 2.1, a = 0[; 
and 

a family TZ of subsets of O t 0 ’ 00 ), directed upwards with respect to union i.e. if A,C G TZ then 

TZ = {{p G {0, 1}[°’°°) : p is cadlag and for all k G 


2.1 


3D G 7 Z with D D AuC [in Example 
N, J(p) k + 1 — J(p)k > e if J(p)k < oo} : e G (0,oo)}, where J(p)k is the time of the k-th 
jump of the path p (= oo, if there is not one)]. 


Denoting B a ( 0) = {x G M : \x\ < a}, we require that (E 0 o ) oeBa ( 0 ) is P-trivial and if a > 0 that 
O = M. With regard to the random field Y think of, for example, the local times of a Markov 
process, the Brownian sheet, solutions to SPDEs [2] etc. 

Let C consist of precisely all the T-adapted, finite variation right-continuous processes c with 
Co = oo such that c takes values in H for some H G 1Z (H may vary with the choice of c). For each 
c G C, we assume we are also given 


(6) an TL®B(f), oo))-measurable “reward” process Z c taking values in a measurable space (A, A) 
[in Example 2.1, (A, A) = (M,£>(M))/; 

(7) and a TL ® £>([0, oo ))-measurable “penalty” process T c taking values in a measurable space 
(E,£) [in Example 2.1, T c := ( Z c ,t c ) and ( E,£) = (Kx [0,oo),5(M x [0,oo)))/. 


We assume that T c and Z c depend only “path-by-path ” on c in the sense that if c(u) = c'(u), then 
T c (a;) = T c (c j) and Z c (co) = Z c (u>) for wGfi, {c, c'} C C . Finally, we are given 


(8) a measurable function f : A [— 00 , 00 ] and a measurable function g : E —>• [— 00 , 00 ] [in 
Example 2.1, f = Mr is the identity on M and g = K[. 


Informally, the idea is to control the movement in such a random field via a control c G C, 
observing at time t G [0, 00 ) only the values of the field in the interval [ct — a,ct + a] n M. What is 
progressively observed then, under the control c, is the process R c = f ((E t Ct+3: ) a . gBa (o))te[o,oo) taking 
values in the space C(B a ( 0),M) of continuous paths from B a ( 0) —> M endowed with the supremum 
norm || • Hoc (which makes it into a Polish space, since it is separable by the Stone-Weierstrass 
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Theorem; and making R c right- continuous), and the corresponding Borel a-field that coincides with 
the trace S(M)' g>Sa ^| ( 7 (B a (o),R) of the product a-field. An admissible control c is either adapted or 


predictable (we will consider both cases - in Example 2.1 admissible controls are predictable) with 
respect to Q c . Furthermore, given acG C: 

(a) rewards accrue according to the function f of Z c ; 

(b) the speed of movement is penalized according to the function g ofT c ; 

(c) both rewards and penalties are discounted at rate a. 

We now formalize the preceding paragraph. The set C of Definition 3.i \ii) is specified as contain¬ 
ing precisely all cgC' that are adapted (predictable) with respect to the natural filtration (denoted 
Q c ; as in Definition \3.\ (vi)) of the process R c , and satisfying (see the definition of J in the para¬ 
graphfollowing) E [f 0 °° e~ at f~(Zf)dt + J 0 °° e~ at g + (T^_)\dc\t] < oo (where \dc\ is the total variation 
of dc (strictly speaking dc is only defined on T]) and is a finite, signed measure lo¬ 

cally, but the family ((|(fc|)| j g([o,T]))Te[o,oo] admits a unique extension \dc\ to £?([0,oo)) as a o-finite 
measure); Tq_ = f Tq). Clearly the observed information Q c depends in a highly non-trivial way on 
the chosen control c. Remark that, for each c £ C, Q c is a sub-filtration of T. 

Next, the payoff functional J (Definition lfi^(v)) is given by: 


e ~ t f(Zf)dt— I e- at g(T c t _)\dc\ t , c E C. 


J(C) = 1 I --at. 

Jo Jo 

Finally, with regard to Definition \3. 7\ define for any c G C and control time S, 

def 


V(c,S) = {de C : d* =c* }, 


and then let G be any subset of 


G' {control times S such that \/c\/d (d 6 V(c,S) implies that S c = S d and Q% c = Ggd.)}', 

clearly the deterministic times [0, oo) C G 7 . We will see that G' = {control times S such that 
Vc 6 C Vd 6 C ( d sc = c s ° implies that S c = S d )}, as long as (Tt,TL) is Blackwell (which can 
typically be taken to be the case). Indeed, note that each R c is a Polish-space-valued right-continuous 
(in particular, progressively measurable) process, so that Corollary 7.10 together with parts M 
and M of Remark \ 7. 7| apply. Regardless of whether or not (D,R) is Blackwell, however, all the 


provisions of Definitions 3.1, 3.7, and 3.12, are met, and so the example is a coherent control 
system. o 


We now give a generalization of the Poisson search model. Example 2.2 


Example 3.16. Random measure search model. Again the time set is [0,oo) (Definition 
T = [0, oo) ). We are given: 

(1) (D,R,F, P), a filtered probability space; 

(2) n, a locally finite Fo-measurable random measure on (M n ,S(M ri )), with a locally finite num¬ 
ber of atoms; 
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(3) k £ [0, oo], the cost of retirement; 

(4) a discount factor a £ [0, oo); 

(5) a £ [0, oo), the observation radius [in Example 2.2, a = l[. 


We require that the location and the sizes of the atoms of 0)) are P-trivial (for Example 2.2, 
condition on them first and fix knowledge of them). 

We model the time of retirement by sending the control to a coffin state after retirement: let 
d M n be a coffin state, topologise W 1 U {3} so that d is an isolated point, clearly it is a Polish 
space. Let D(M+; M n U {3}) be the space of cadlag maps m : [0, oo) —> U {9}, constant at d after 
first hitting d, and such that Ifom is locally bounded. 

Let C be the collection of T-adapted random elements c with values in D(M + ; M n U{<9}) satisfying 

co = 0 . . 

and 


Of course we take T c = f LI and P c = f P for each c £ C (Definitions 3.1 (in) 


3.i (iv)). 


To allow for extra conditions on controls (of a pathwise nature) let 1Z be a family of subsets of 
D(M + ;M n U {5}), directed upwards with respect to union [in Example 2.2, take 


LI = {{p £ -D(R_|_; M n U {0}) such that sup ||p(£)||r« < 1}}, 

t<T(p) 

where r(p) is the first hitting time of d by the path p. To allow instead controls that, for example, 
are bounded up to explosion (uniformly in u) by some constant that depends on the control, we 
would take 7Z = {{p £ Z)(M + ;M Tl U {9}) such that sup t<T ( p ) ||p(i)||iRn < k} : k £ (0,oo)}./ 

The corresponding “controlled process’’ is X c given by 

X( = [ c s ds + aW t , 

Jo 

where 

(6) W is an JF-Brownian Motion and a £ M. 

Now for what we observe. We observe the BM, W, and the location and sizes of the atoms of p 
in a ball of radius a around X c . Note that we don’t observe the whole measure in this ball. In the 
Poisson special case, this is complete observation. 

To define a space in which the observations will live, for d £ No let S d be the quotient under 
permutation of (M n X [0, oo)) d (setting (M n x [0, oo))° = {0 }) endowed with the quotiented Euclidean 
metric. Now define 0 = M n x Ud£N 0 {d} x S d , where the metric on the disjoint union UdeN 0 {d} x S d 
is defined by taking the minimum of the metric on each fiber ({d} x S d ) with 1, and making the 
distance between members of different fibers 1. It is easy to check that 0 is a Polish space. 

We think of d as telling us how many atoms we are currently seeing around a location, and then 
S d gives the locations and masses. The factor M n corresponds to observing W. 

Now endow 0 with the corresponding Borel measurable structure, T, which coincides with the 
product measurable structure ofW 1 and of the disjoint union Ud e pj 0 {d} x S d . 
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So what we observe, given c E C is the ( Q,T)-valued process O c = ( W,(D C ,M C )), where at 
time t E [0, oo), D] is the number of atoms of p in B a (Xf), and then Mf is the location and sizes 
of the atoms of n in B a (Xf). Note that O] is Ft-measurable for t E [0,oo) [IQj fa. 19, Lemma 2.1]. 
We set Q c to be the natural filtration of O c (Definition 3.l(vi))). Then C of Definition 
the set of those c E C' that are 


is 


(i) predictable with respect to Q c 

and such that 

(ii) there is an H E 71 (that may depend on c) with c taking values in H 

and 

(in) E p f 0 T e~ at pi(B a (Xf))dt + Ke~ aTC t(r c < oo) < oo, where t c is time of retirement of c (i.e. 
the first hitting time of d by c). 


For c E C, we set 


J(c ) d = - 


e~ at p,(B a (Xf))dt - Ke~ ar 1 (r c < oo) 


(Definition \3. \ (v)\ ). Finally, we define V(c,S), G and G ' exactly as in Example 3.15. Since the 
process O c is right-continuous and Polish-space-valued, the same remark applies concerning V{c,S), 
as in Example \3 .1 5\ o 


4. The conditional payoff and the Bellman system 

Definition 4.1 (Conditional payoff and Bellman system). For c E C and S E G, we define: 

J(c,S) d = E pc [J(c)|^ c ], and then V(c,S ) d = P c |gc c -esssup de2?(Cj5) J(d,5); 

and say c E C is conditionally optimal at S E G, if V(c,S) = J(c,S ) P c -a.s. The collection 
(J(c, 5))( C)l s)eCxG is called the conditional payoff system and (V(c, S))( c ,s)<eCxG the Bellman 
system. 


Remark 4.2. 


(i) Thanks to Definition 3.12, the essential suprema appearing in the definition of the condi¬ 
tional payoff system are well-defined (a.s.). 

(ii) Also, thanks to Condition 3.7 4), V{c, S) only depends on S through S c , in the sense that 
V(c, S) = V (c, T) as soon as S c = T c {P c -a.s.}. Clearly the same holds true (trivially) of 
the system J. 


Some further properties of the systems V and J follow. First, 

Proposition 4.3. V(c, S) is Q^ c -measurable and its negative part is P c -integrable for each ( c,S ) E 
C x G. Moreover if c d, then V(c,S ) = V(d,S ) P c -a.s. and P d -a.s. 

Proof. Measurability of V(c, S) follows from its definition. Moreover, since each D(c,S ) is non¬ 
empty, the integrability condition on the negative parts of V is also immediate. The last claim 
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follows from the fact that V(c,S ) = U(d,S) (adaptive dynamics) and P c | go = P d \ G d (stability 

s V s d 

under stopping), when c d. □ 

Second, Proposition |4.6[ will 


below, 

and 

(ii) give sufficient conditions for the equality J(c,S) = J(d,S) to hold P c and P^-a.s. on an 
event A £ Q^ c , when c d (addressing the situation when the two controls c and d 
agree “for all times” on A). Some auxiliary concepts are needed for this; they are given in 
Definition 14.51 

Definition 4.4 (Payoff system). A collection X = (X(c,T))( c ^)eCxG °f functions from [—oo,oo]^ 
is a payoff" system with respect to (C, G), if 

(i) X(c,T ) is C/^c-measurable for all (c,T) £ C x G 
and 

(ii) X(c,S) = X(c,T) P c -a.s. on the event {5 C = T c }, for all c £ C and {S,T} C G. 
Definition 4.5 (Times accessing infinity). 

(1) For a sequence (t n )neN of elements of [0, oo], we say it accesses infinity, if sup ngN t n = oo. 

(2) If P is a probability measure on a sample space $, 4 C $, S„ : $ A [0, oo] for n £ N, and 
(S n (u)) n£ ^ accesses infinity for (respectively P-almost) every u £ A, then we say ( S n ) n& ^ 

accesses infinity pointwise (respectively P-a.s.) on A. 

In the context of a stochastic control system: 

(3) If (S n ) n is a sequence in G, A C fl, c £ C and (5^(w)) n£ N accesses infinity for {P c -almost} 
every a; £ A then we say that (<S n ) accesses infinity {a.s.} on A for the control c. 


establish that (J(c,5))( Cjl s)ecxG is a (C,G) payoff system in the sense of Definition 


4.4 


Proposition 4.6. (J(c,<S))( C)5 ) eCxG is a (C,G) payoff system. 

Moreover, given c,d £ C, S £ Q and A £ Q$ c with c ~s d, if 

(i) there exists a sequence (5 n ) nS N from Q which is {a.s.} nondecreasing and accesses infinity 
{a.s.} on A for both of the controls c and d, and for which c d and A £ Q c Sa for each 
n £ N; 

and 

(ii) E pc [J{c)\Q c 00 ] = P pd [J(d)\Gi 0 } P c -a.s. andP d -a.s. on A; 
then J(c,S ) = J(d,S) P c -a.s. and P d -a.s. on A. 


Proof. By definition, J(c,T) is ^ c -measurable. 
F £ Qg c fl Q'fic and so J(c, T) = J(c,S ) P c -a.s. 
(J(c,S))( C)5)eCxG is a (C, G) payoff system. 


iking c £ C and setting F c = {T c = S c }, 
F (applying Lemma A.l), establishing that 
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To show that J(c, <S) = J(d,S) P c -a.s. (and then, by symmetry, that P f/ -a.s.) on A under 


(4.1) 


conditions (i) and |(ii)[ we need only establish that 

l A E pC [J(c)\g c Sc \ = t A E pd [J(d)\g* d \ P c -a.s. 

Now 


(i) 


and Lemma 


A.4 


tell us that P d \gd and P c |gg o agree on A, so that, taking an arbitrary 


B £ Q c Sc = Q% d 

E pc [E-[J(d)|C]l A l s 
hence E pc [E pc [J(c)|^]l A l B 
so E pc [J(c)l A l 


E h 

E f 


[E p [J(d)|^]l A l 


B 


E K [J(d)|^]l A l B ] (by assumption [(ii)|), 

B\ = E pd [E pd [J(d)l A |^ d ]l B ] 

(by standard properties of conditional expectation) 

and thus E pc [J(c)l A l B ] = E pc [E pd [J(d)l A |<^ d ]l B ] 

(since P c \g% c = P d \ G d by stability under stopping). 
5 y s d 


Since B is arbitrary and A £ G%c, we conclude that (4.1) holds. 


□ 


5. Bellman’s principle 


Definition 5.1 ((super, sub-)martingale systems). A collection X = (X(c, <S))( c ,s)e(c,G) °f func¬ 
tions from [— 00 , 00 ]^ is a (C, G)-martingale (respectively supermartingale, submartingale) 
system, if for each (c, S) € C x G 

(i) X(c,S) is t?£ C -measurable, 

(ii) X(c,S ) = X(d,S) P c -a.s. and P d -a.s., whenever c ~s d, 

(iii) X(c,S) is integrable (respectively the negative, positive part of X(c,S ) is integrable). 
and 

(iv) for all {5,T} C G and c G C with S d < T d {P d -a.s.} for d €E T>(c,T), 

E pc [X(c,T)\g%} = X{c,S ), (respectively E pc [X(c, T)\G c S c] <, > X(c,S)) P c -a.s. 

In order to be able to conclude the supermartingale property of the Bellman system (Bellman’s 
principle), we shall need to make a further assumption. 

Definition 5.2 (ULP). Fix e £ [0, 00 ), M £ (0, 00 ], c £ C and S £ G. We say that (J(d, 5))dex>( c ,s) 
has the (e, M)-upwards-lattice property if, whenever {d, d'} C T>(c,S), there exists a d" £ 
T>(c, S) such that 

J(d",S) > (M A J(d,S)) V (M A J(d',S)) — e P c -a.s. 

Assumption 5.3 (Weak upwards lattice property). For every c £ C, S £ G and {e, M} C (0, 00 ), 
(J(d, <S))d 6 D( Ci 5 ) has the (e, M)-upwards-lattice property. 

We shall make it explicit in the sequel when this assumption is in effect. 

The following theorem gives some conditions which guarantee Assumption |5.3| holds. 
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Theorem 5.4. Let c E C, S E G and e E [0, oo), M E (0, oo]. Then Condition (Cl)=$ ( C2)=$ \(C3) 
where 

(Cl) (i) For all d E V{c,S). P d = P c 

and (ii) For all {d, d'} C V{c,S) and G E Q% c , there is a d" E V(c,S) such that J{d") > 
M A [1 G J{d) + 1 n\ G J(d')\ - e P c -a.s. 

(C2) For all {d, d'} C V(c,S ) and G E Q% c , there is a d" E V(c,S) such that 
J(d", S)>M A [1 G J(d, 5) + In \ G J(<t, S )] - e P c -a.s. 

(C3) {J{d,S))^v(c,S) has the (e, M)-upwards-lattice property. 


Proof. Implication |(C1) => (C2) follows by conditioning on Q'f c under P c . Implication (C2)=>(C3) 


follows by taking G = { J(d,S ) > J(d',S)} E £/j 


Remark 5.5. 


□ 


The upwards lattice property of Assumption 5.3 represents a direct connection between 
((P c ) ce c, J) on the one hand and ((<7 c ) c eCi G, (£>(c, <S)) ce c,5eG) on the other. It is weaker 
than insisting that every system (J(c,<S)) ce c be upwards-directed (i.e. having the (0,oo)- 
upwards lattice property), but still sufficient to allow one to conclude Bellman’s (su- 


per)martingale principle (Theorem 5.7). 


(ii) A more precise understanding of the relationship between the applicability of Bellman’s 
principle, and the linkage between ((P c ) ce c, J) and ((f/ c ) c ec, G, (D(c, 5)) ce c,SeG) remains 
open. In particular, as we have seen in Example |3. 1 1| , it appears that whether or not 
a control time can feature as a member of G (whilst maintaining Bellman’s principle) is 
related to whether or not the act of controlling can be effected at that time. 

(iii) It may be assumed without loss of generality (in the sense which follows) that {0, oo} C G. 
Specifically, we can always simply extend the family T>. by defining T>{c. oo) = f |c| and 


def 


V(c, 0) = C for each c E C - neither the provisions of Section 3.1 (Definitions 


3.1 


3.7 


and 3.12) nor the validity of Assumption 5.3 being affected. 


iv) Assumption 5.3 is of course trivially satisfied when the filtrations Q all consist of (proba¬ 


bilistically) trivial u-fields. 


Example 3.15 continued. We verify that in Example \3.15\ if the base ( fh F) is Blackwell or the 
members of G are deterministic, then Property \(Cl )| from Theorem \5.4\ holds with M = oo, e = 0. 

Let ci C 2 , A E Qgc, = Gg%- It is enough to show that c = f cil^ + C 2 ln\T E C (since then 
we will have J(c ) = J(ci)l ,4 + J(c2)1q\^, thanks to the “path-by-path v dependence ofT c and Z c 
on c; and clearly c ci,C 2 j. The control c is a right-continuous finite variation O-valued process 
with initial value oq, satisfying the requisite integrability condition on and g + . Also, if ci takes 
values in H, H E PL and C 2 takes values in H', H' E PL, then since PL is upwards directed with 
respect to union, there is an H" E PL with H" D H U FT, and clearly c takes values in H". 
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Noting that the filtration Q c is included in the filtration T, it remains to check that c is Q c -adapted 

def 

(predictable). Setting P = S Cl = S C2 , it is sufficient to argue that d[o,p] = cilmpi = C2l[o,p], 
cI^p^Ia = cil^ooplA and cldP i 00 |)ln\A = C 2 ldP,oo[)lD\A are all Q c -adapted (predictable ). 

To this end, note that P is a stopping time of Q C1 and of Q C2 , and ( c\) p = c p = (c2) p , 
hence (R c ') p = ( R c ) p = (R C2 ) P . Then G c t \ {t < P} = Gf%< P} = G?\{t<P}, whilst Gf\ {P<t}nA = 
Gf 1 \{P<t}nA and Gf\{ P<t }n(n\A) = G? \{P<t}n(Q\A)- Also, by Theorem 


7.9 


Proposition 


7.8 


and Proposition 7.12 to follow (or trivially when P is deterministic), all the events {t < P}, 
(Q\A)n{P < t} andAn{P < t} belong to o((R Cl ) PAt ) = o((R C2 ) PM ) = o((R c ) PM ) C C7((R C Y) = 
Qf (and, of course, Qf 1 n Qf 2 ) for all t G [0, oo). 

Now apply Lemma A.8| o 

Example |3.16 continued. Using the same method as above, one verifies that, in Example \3.16\ 
if the base (Ll,P) is Blackwell or the members of G are deterministic, then again Property \(C1 )\ 
from Theorem \5.4\ holds with M = oo, e = 0. We leave the details to the reader. o 


Theorem 5.6. [Cf. [SJ p. 94, Lemma 1.14]./ Under Assumption \ 5. for any c G C, T £ G and 
any sub-cr-field A of Q pc : 


In particular, 


ePC [E(c,T)|A] = P c U-esssup deC(CiT) E p ' [J(d)\A] P c -a.s. 


E pc V(c,T)= sup E pd J(d). 

deV(c,T) 


(5.1) 


Proof. By Lemma A.7 we have, P c -a.s.: 

E pc [V(c,T)\A] = P c U-esssup deI)(C:r) E pc [E pd [J(d)|^ d ]|A] 

= P c U-esssup deC(CiT) E pd [E pd [J(d)|gf c ]|A], since <7f c = Q* d and P c | g^ c = P d \g^ d , 


for d c, from which the claim follows. 


□ 


Theorem 5.7 (Bellman’s principle). Suppose that {0,oo} C G. Under Assumption 5.3: 

(Bl) (V(c, <S))( c ,s)eCxG is a (C,G)- supermartingale system. 

(B2) If c* G C is optimal, then (V(c*,T))tc g has a constant P c *-expectation (equal to the 

* 

optimal value v = E pc J(c*)). 

pc* 

(B3) If c* G C is optimal and E K J(c*) < oo, then (V(c*,T)) t^g is a Q-martingale in the 
sense that 

(i) for each T G G, V(c*,T) is G pc * -measurable and P c -integrable 
and 

(ii) for any {5, T} C G with S d < T d {P d -a.s.} for d G V(c*,T), 

E pc y(c*,T)|S c **] = V(c*,S) P c * -a.s. 
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(Bf) If c* € C is conditionally optimal at S £ G and E pc J(c*) < oo, then c* is conditionally 

optimal at T for any T G G satisfying T d > 5 c/ {P rf -a.s.} for d G T>(c*, T). In particular, 

* 

if c* is optimal, then it is conditionally optimal at 0, so that if also E p J(c*) < oo, then c* 
must be conditionally optimal at any S G G. 

Regardless of whether or not Assumption \5.3\ holds: 

(B5) If c* G C and G includes a sequence (S n )neN 0 for which 

(i) S 0 = 0, 

(ii) the family (V(c*,S n )) n >o has a constant P c *-expectation and is uniformly integrable, 
and 

(Hi) V(c*, S n )—> V(c*, oo), P c *-a.s. (or even just in P' -probability), as n ^ oo, 
then c* is optimal. 


Remark 5.8. Recall that the assumption {0,oo} C G is innocuous (see Remark 5.1: |pii) ). 

Proof. Let {5, T}cG and c G C with S d < T d {P d -a.s.} for d G D(c, T). Then, since S c < T c {P c 


a.s.}, Qg c C £f c , and (by adaptive dynamics) T>(c, T ) C T>(c, S), so that we obtain via Theorem 5.6 
P c -a.s., 

E pc [V(c,TW Sc ] = P c b| c -esssu Pde2?(C)r) E pd [J(d)|^c] 


(since Q c sc = Q d d for S rsJ ffy d) which, together with Proposition 


< pC |e|c- esssu P deC(c,5) EP1 [J(d)\Gsc] = V(c,S) 

establishes 


4.3 


(Bl) 


Assertion 


|(B2) follows at once from ( |5.1[ ). To establish the martingale property |(B3)| let c* be optimal and 
{5,T} C G with S d < T d {P rf -a.s.} for d G V(c*,T). Note that by the supermartingale property, 
v = E p ’ E pc [V(c*, T)\Gg c *] < E p " V’(c*,«S) = v. So, if v < oo, we conclude that V(c*,T) is 
P c *-integrable, and the martingale property follows. 

Now if c* is conditionally optimal at S , E pt J(c*) < oo, and S d < T d {P d -a.s.} for d G V(c*,T), 
then since V is a (C, G)-supermartingale system, E p ‘ J(c*) = E pL J(c*,S ) = E pc V(c*,S) > 
E pt V(c*,T). On the other hand, for sure, V(c*,T ) > J(c*,T), P c *-a.s., so E pt V(c*,T) > 
E p " J(c*,T) = E p J(c*) hence we must have V{c*,T) = J(c*,T), P c *-a.s., i.e. c* is conditionally 
optimal at T. So (B4)| holds. 


Finally, notice that, under the assumptions in (B5) V (■ c*,S n ) —> V (c*, oo) in L 1 (P C *), as n -» oo, 
and so 


v — sup E P J (c) = E P V (c*, 0) = E p V(c*,S n ) 

esc 


n—>oo r-P c 

-> E 


V (c*,oo) = E pc J(c*) 


□ 


Theorem 5.9 (Supermartingale envelope). Under Assumption 5.3. V is the minimal (C,G)- 
supermartingale system W satisfying the terminal condition 

W(c, oo) > E p [J(c)\g^\ P c -a.s. for each c G C. 


(5.2) 
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Proof. That V is a (C, G)-supermartingale system satisfying (5.2) is clear from the definition of V 
and Theorem 5.7 Next, let W be a (C, G)-supermartingale system satisfying (5.2). Then for all 
(c, T) G C x G and d G V(c, T), 

lT(c,T) = W(d,T) > E pd [fT(d,oo)|^ d ] > E pd [E pd [J(d)|^]|^ d ] = J(d,T) P c -a.s. and P d -a.s. 
Thus W(c, T) > E(c, T), P c -a.s. □ 


Remark 5.10. Notice that we recover from Theorem 5.9 the Snell envelope characterisation of 
solutions to the problem of optimally stopping a cadlag process X adapted to a filtration T = 
(Xt)teT by taking C to be the collection of ^-stopping times, r, and setting, for t & T. QJ = Ft/\ r - 

Example 3.11 continued. In order to complete the argument in Example 3.11\ we establish 
the relevant lattice property. For arbitrary X G C, n G No, G G Gs n , and {Y, Z} C C with 
ySn — x Sn = Z Sn , we find that the control U, given by 

U t = itl[ 0)Sn ](t) + Y t l iSn ,oo)(t)lG + z t^(s n ,oo){t) 1 G c , 

which coincides with Y and Z on [0, S n ] and then with Y on G and Z on G c strictly after 
S n , is conditionally admissible at time S n for X. By Theorem 5.4 1 (Cl) we see that the fam¬ 
ily {E p [J(Y)\g s J : Y € C,Y Sn = X s »} is directed downwards for each n G Nq. Hence, 


def 


5.7 


def 


with G = {Sn : n G No U {oo}} and with 


setting Soo = oo, one can apply Theorem 
V{X, S n ) = f {Ye C : Y Sn = X s "} for X G C and n G N 0 U {oo}. 

We note that it also follows from Theorem 5.7 and (1) on p. [To[ that if G contains a strictly 
positive finite deterministic time t (with the equivalence class T>(c, t ) being defined in the obvious 
way), then Assumption 5.3 must fail. o 


6. A SOLVED EXAMPLE 


We now conclude Example 2.1 Recall the notation of section [3] and Example 2.1 (as continued 


on 


pp.[7|and|ll[) and note that all the conditions of section[3]are satisfied. Recall also that Example 


2.1| is a special case of Example 3.15, which (see p. 19) satisfies the weak upwards lattice property 
(Assumption |5.3} , provided (11,77) is Blackwell or G contains only deterministic times. 

Now we shall give the solution in some fairly straightforward cases. 


Proposition 6.1. 

(a) If K(z,t) = —2z/a, ( z,t) G R x [0,oo) ; then the optimal payoff, V satisfies V(x) = x/a 
and any control achieves it. 

(b) If K has the form: 


K(z,t)m 


— r\ — Irl p-rWiu-z\ _ p— r y\ z \ 

—----- H-I 4 >(u)du + l(o,00 )(z)L(z,t), (. z,t ) fix [ 0 ,00), 

a a7 ' 
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where 7 = \f2ct, <ft is the standard normal density and L(z , t) is nonnegative, measurable, 
and of polynomial growth (uniformly in t), then V(x) is a symmetric function of the pa¬ 
rameter x. Moreover, letting c e be the control which waits an amount of time e both after 
each time it jumps and at time zero, and thereafter jumps at the first entrance time of Z c 
into (—oo,0], for any such K, E p J(c e ) —tV(x) = 71 1 , as e | 0. 

Remark 6.2. We see here, in the jump times of c e , an example of a whole sequence of non-control- 
constant control times (members of G r , provided (Q,'H) is Blackwell). 


According to Bellman’s principle (Theorem 5.7) and the strong Markov property, for each c E C, 
the following process (where V is, by a slight abuse of notation, the value functioiQ: 


£>c def 


s Z c a ds - 


e- as K(Z c s _,Tf_)\dc s \+e~ at V(Zf,Tf), 


( 6 . 1 ) 


Jo J(0,t] 

should be a ( Q c , P)-supermartingale (in t E [0, 00 )). Moreover, if an optimal strategy c* exists, then 
S c should be a ( Q c , P)-martingale (or, when dealing with a sequence/net of optimizing controls, 
the corresponding processes should, in expectation, ‘be increasingly close to being one’). 

This leads us to frame the following: 


Lemma 6.3 (Verification Lemma). Suppose that 
UP the function K(z,t ) has uniformly (in t) polynomial growth in z, 
and that the function h : R — > R satisfies 

V(i) h is of class C 1 and is twice differentiable, with a second derivative, which is continuous 
except possibly at finitely many points, where the left and right derivatives exist and are 
continuous from the left, respectively right; 

V(ii) h, h! and h" have polynomial growth. 

Let l E R and suppose that: 

W(i) z — ah(z ) + \h"{z) < 0, for a.e. z E R; 

W(ii) — K(z,t ) + f R [h(-\ftu — z) — h{z)\<j)(u)du < 0, for all z E M, f E [0,oo); 

W(iii) z — ah{z) + \h"(z) = 0, for a.e. z > —l; 

W(iv) —I\(z, t ) + J R [h(y/tu — z) — h(z)\(f(u)du = 0, for all z < —l, t E (0, 00 ). 

Then V = h and an optimizing (as e f Oj net of optimal controls is (c e ) e >0 where c e is the control 
which waits for a period of time e after each jump of c e and also at the start, and thereafter switches 
the observed Brownian motion precisely at the first entrance time of Z cL into the set (— 00 ,—/]. 
Remark that c e is previsible with respect to Q cL . 


Proof. For each control c, define the process S c by 

S( = f f e ~ as Z c s ds- [ e~ as K(Z c s _,Tf_)\dc\ s + e~ at h(Zf) 

Jo J(o,t] 

^More precisely, for z E R, u € [0, 00 ), V{z,u ) is the optimal payoff of the related optimal control problem in 
which, ceteris paribus, B 1 = z + H u +., for a Brownian motion H independent of B°. 
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(as in (6.1)). 

The semimartingale decomposition of S c may then be effected relative to the completed measure 
P and the usual augmentation G c + of Q c , with respect to which Z c is a semimartingale (indeed, 
its jump part is clearly of finite variation, whilst its continuous part is, in fact, a Brownian motion 
relative to the augmentation of the natural filtration of Thanks tcV(i) and V(ii) we 


obtain, by the Ito-Tanaka-Meyer formula [T% , p. 214, Theorem IV.70, p. 216, Corollary IV.1], 
P-a.s. for all t E [0, oo): 

5 t c = h(x)+ f e~ as Z c s _ds+ [ e~ as (—K(Zg_,Tg_))\dc\ s + f e~ as (-a)h(Z c s _)ds (6.2) 
Jo 4 ( 04 ] Jo 

s ✓ ^ ^ 


defpj 


4| f D i 


d =C 2 


(1) 


+ / e~ as h'(Z c s _)dZ c s + / e-“- -h"{Z c s _)d[Z c ]f* + £ e 


0 <s<t 


d =c 3 


( 2 ) 


(1) 


Ah(Z c s )-h'(Z c s _)AZ c s 


Note that the parts labelled (1) add to 


(i) 


e~ as ti(Z c s _)d{Z c )? s , 


(6.3) 


d ^M i 


which is a {G c +, P)-martingale in f E [0, oo) (since \Z C \ < B° + B 1 ). On the other hand, the 
compensator of the term labelled (2) is 


( 2 ) 


[ \dc\ s e~ as 

f du<t>{u) (h(y/r^Zu - Z c s _) - h{Z c s _)) 

'(04] 

Um. J 


(6.4) 


making 


( 2 ) 


'(041 


\dc\ s e 


Ah(Z c s )~ / ducf>(u) (h(yfi*;u-ZS_)-h(ZS_)) 


(6.5) 


d = M 2 


into a (G c , P)-martingale (in t E [0, oo)). For, if t is a predictable stopping time with respect to 
some filtration (in continuous time) Z, U is a ^-measurable random variable, and Q a probability 
measure with Q[|C/| 1 [ 0 ,t] ° t] < oo for each t E [0, oo), then the compensator of I71 | T]00 d (relative to 
(Z, Q)) is Q[t/|2 r _]l| r oo |). This fact may be applied to each jump time of the t? c -predictable process 
c (since \Z C \ < B° + B 1 ). Then linearity allows us to conclude that M 2 is a (G c , P)-martingale 
(thanks to the ‘e-separation’ of the jumps of c). 





























ON THE INFORMATIONAL STRUCTURE IN OPTIMAL DYNAMIC STOCHASTIC CONTROL 


25 


Note that the properties of being a cadlag (super)martingale [HI p. 173, Lemma 11.67.10] 


or 


predictable process (of finite variation) are preserved when passing to the usual augmentation of 
a filtered probability space. Therefore it follows that, relative to {G c +, P), M = M 1 + M 2 is a 
martingale, whilst D == D 1 + D 2 (respectively C = C 1 + C 2 + C 3 ) is a pure-jump (respectively 


continuous) predictable process of finite variation. Now conditions W(i) and W(ii) show that 


C and D are nonincreasing and so we have obtained the Doob-Meyer decomposition of S c = 
h{x) + M + D + C (P p. 32, Corollary 3.16] [lT[ p. 412, Theorem 22.5]), and S c is a (G c +, P)- 
supermartingale^ 

Assumptions 


V(h) 


and 


UP 


ensure that Sf J(c) '= and so we conclude from Theorem 


5.9 


(applied with W = S and G = [0, oo]) that 


h > V. 


To establish the reverse inequality, we will show that E p J(c £ ) —>■ h(x) as e I 0. 

£jl 

First, note that since S c J(c) it is sufficient to check that C' 00 (c e ) and ^^(c 6 ), the limiting 
values of the continuous and pure-jump components in the Doob-Meyer decomposition of S C<L , 


converge to 0 in L 1 as e I 0. Now from W(iii) 


r°° 1 

Coo(c e ) = J Q e~ as [Zf — ah(Zf) + -h"(Zf)]ds 


e~ as [Zf - ah(Zf) + -h"(Zf)\ o Zfds 


and from V(ii) we see that to show that this converges in L 1 to 0 it is sufficient, by the Dominated 
Convergence Theorem, to show that l(_oo,-a) ° -^ cC |[o,T] —> 0, in P x Leb|g([o,r])-measure for each 
T E [0, oo) and a E [l, oo), which is clear from the definition of c e . 

Turning to the jump part, 


Doo(c e ) = I e 

I o 


/ 6 °\ Jr du ^ U ^ \ h (\f^- u ~ z f-)~ H z s-) 


-K(Zf_,rf_))\dc\ s , 


and so from W(iv) it follows that 


f c 

Doo(c e ) = / 

Jo 


ducj)(u ) ( h{Jrf_u — Z^_) — h(Z^_) 


~ K(Z^_, Tg_) | l ( _/ >oo) o Z c s _\dc\, 


and since c e never jumps when Z cC E (—Z,oo), we see that ^^(c 6 ) = 0. 


□ 


It is now quite straightforward to prove Proposition 6.1 


Proof of Proposition 6.1 


a) Recall that in this case K(z,t) = —2z/a. Taking h(z) = z/a. for z E M, we see that |W (i~)| 


|W(ii) W(iii) W(iv) are all satisfied with equality everywhere, without the qualifications involving 


the boundary l. Taking expectations in (6.2), and passing to the limit as t —> oo via dominated 


convergence, we see that h is the optimal payoff, and any control from C realizes it. 
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(b) 


def 


Set l = 0 and h(z ) = if(\z\), where 'ip(z) = 


def 'y z-\-e 1Z 


aj 


z e 


For such an h, h is C 2 


is satisfied with strict inequality on z E (—oo,0); and |W(ihj| is satisfied on [0, oo). Then 


m iv)| and |W(ii)| are satisfied for K of the form specified in |(b) 
Lemma 16.31 


on p. 


22 and the result follows by 

□ 


7. Stopping times, stopped processes and natural filtrations at stopping times - 

INFORMATIONAL CONSISTENCY 


7.1. The nature of information. We now turn our attention to Question |1.6[ We shall investi¬ 
gate: (i) the precise relationship between the sigma-fields of the stopped processes and the natural 
filtrations of the processes at these stopping times, and (ii) the nature of the stopping times of the 
processes and of the stopped processes, themselves. Here is an informal statement of the kind of 
results that we will establish ( T x denotes the natural filtration of a process X ): 

If X is a process, and S a time, then S' is a stopping time of T x , if and only if it 
is a stopping time of J- xS . When this is so, then J- x = <r(X' s ). In particular, if 
X and Y are two processes, and S is a stopping time of either T x or of T Y , with 
X s = Y s , then S is a stopping time of J~ x and J~ y both, moreover J- x = crfW 5 ) = 
a(Y s ) = Xg . Further, ii U < V are two stopping times of X x , X again being a 
process, then cr(X u ) = T x C F x = a(X v ). 

We will perform this study of the nature of information generated by processes in the ‘measure- 
theoretic’ and then the ‘probabilistic’ setting. This will mirror the parallel development of the two 
frameworks for stochastic control from the preceding sections. 

The main findings of this section are as follows: 


in the ‘measure-theoretic’ case, Lemma 7.5, Proposition 7.8 Theorem 7.9, Theorem 7.10 


and Proposition 7.12 


• for the case with completions, Corollaries 7.14 and 7.15 (in discrete time) and Proposi¬ 
tion 7.19, Corollaries 7.20, 7.22 and 7.23 (in continuous time). 

We have already referenced many of these results in the preceding sections. 

It emerges that everything that intuitively ought to hold, does hold, if either the time domain 
is discrete, or else the underlying space is Blackwell (and, when dealing with completions, the 
stopping times are predictable and the filtration quasi-left-continuous; but see the negative results 
of Examples 7.16 and 7.17). While we have not been able to drop the “Blackwell assumption”, we 
believe many of the results should still hold true under weaker conditions - this remains open. 


7.2. The ‘measure-theoretic’ case. We begin with some relevant definitions. Indeed, the defini¬ 
tions to be introduced presently are (mostly) standard [Till3l passim]: we provide their definitions 
explicitly in order to avoid any ambiguity, to fix notation, and to recall some measure-theoretic 
facts along the way. 
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Throughout the remainder of this section T = No or T = [0, oo), Gl is a set and {E, £) is a 
measurable space. When t G T = No, [0 ,t\ denotes the set {0, A time (on fl) is a map 

Gi —> S U {oo}. Recall also that 2 X denotes the power set of a set X and a (/) denotes the er-field 
generated by a map / (the measurable structure on the codomain being understood from context). 

By a process (on Gl, with time domain T and values in E), we mean a collection X = (Xt)teT 
of functions from Gl into E. With T x = f a(X s : s E [0,f]) for t G T, E x = f {X x ) t ^T is the 
natural filtration of X. Remark that, for every t G T, E x = <r(X|[o,t]), where for w E Q, 
A|[ 0j t](o;) = f (X s (u;)) se [o,t]) -X”|[o,t] : ^ > E^^ is an E x -measurable map. For w 6 fl, the 

w-sample path of X, X(uj), is the function T 3 t t— > Xf (oj) G E. In this sense X may of course 
be viewed as an E x /f^-measurable map, indeed J~ x = cr(X). Then ImA will denote the range 
(image) of the function X : —> E T . Henceforth, unless otherwise made clear, we will consider a 
process as a map from into E T . 

If S : — > T U {oo} is a time, the stopped process X s is defined by 

X t s M = X sh M, w G si, t g t. 

If further Q is a filtration (i.e. a nondecreasing family of cr-fields indexed by T) on fl and S is 
t/-stopping time (i.e. {S <t} E Qt for all t G T), then 

Qs = f { A G Goo : A (1 {S < t} £ Gt. for each t G T} 

is the filtration G at (the time) S. Note that, if T = No, if X is a (/-adapted process and if S is 

a (/-stopping time, then X s is automatically adapted to the stopped filtration ( GnAs)neN 0 ■ F° r ’ if 
n G N 0 and Z G S, then (X^y^Z) = (u ^X^(Z) n {S = m}) U (X-^Z) n {n < 5}) G Gsau- 

On the other hand, in continuous time, when T = [0, oo), if X is (/-progressively measurable and if 

S is a (/-stopping time, then X s is also adapted to the stopped filtration ((//As)ie[o,oo) (and is G- 
progressively measurable) [121 p. 9, Proposition 2.18]. Note also that every right- or left-continuous 
metric space-valued (/-adapted process is automatically (/-progressively measurable (limits of metric 
space-valued measurable functions being measurable). 

Next, for a a- field E on Q, define an equivalence relation ~ on fi, via 

(a; ~ oj') (for all A G X, 1 a(w) = 1a (c*/))- 

Then define the atoms of (f i,E) to be the equivalence classes of Gl under ~. The Hausdorff 
space of (Gl,E) is the corresponding quotient space under ~. 

The measurable space (Gl, F) is said to be 

(i) separable or countably generated, when it admits a countable generating set; 

(ii) Hausdorff, or separated, when its atoms are the singletons of Gl [3 P- 10]; 

and 

(iii) Blackwell when its associated Hausdorff space is Souslin |3), p. 50, III.24], 
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Furthermore, a Souslin space is a measurable space, which is Borel isomorphic to a Souslin topolog¬ 
ical space. The latter in turn is a Hausdorff topological space, which is also a continuous image of 
a Polish space (i.e. of a completely metrizable separable topological space). Every Souslin measur¬ 
able space is necessarily separable and separated. O p. 46, III. 16; p. 76, III.67] For a measurable 
space, clearly being Souslin is equivalent to being simultaneously Blackwell and Hausdorff. 

The key result for us is Blackwell’s Theorem [31 p. 51 Theorem III.26] (repeated here for the 
reader’s convenience - we shall use it repeatedly): 

Theorem 7.1 (Blackwell’s Theorem). Let be a Blackwell space, Q a sub-a-field of T and 

S a separable sub-cr-field of T. Then Q C S, if and only if every atom of Q is a union of atoms of 
S. In particular, an T-measurable real function g is S-measurable, if and only if g is constant on 
every atom of S. □ 

Some elementary observations that we shall use without special reference are gathered in 

Lemma 7.2. 

(i) If Y is a mapping from A into some Hausdorff (respectively separable) measurable space 
(B,B), then Y is constant on the atoms of cr(Y) (respectively cr(Y) is separable). 

(ii) Conversely, ifY is a surjective mapping from A onto some measurable space (B,J3), con¬ 
stant on the atoms of a(Y), then {B,B) is Hausdorff. 

(Hi) Any measurable subspace (with the trace cr-field) of a separable (respectively Hausdorff) 
space is separable (respectively Hausdorff). 

(iv) If f : A -» (B,B) is any map into a measurable space, then the atoms of a(f) always 
‘respect’ the equivalence relation induced by f, i.e., for {cu,u/} C A, if /(w) = f{oj'), then 
vj and uj' belong to the same atom of a(f). 

Proof. 

(i) Suppose that ( B , B) is Hausdorff and A is an atom of a (T) containing uq and uq. If F (wi) f 
Y(uj 2 ), then there is a W G B with %(F(wi)) / 1 w(T(w 2)), hence ly-i(iv)(wi) / 

a contradiction. 

(ii) If { b , b'} C B and 1 w(b) = 1 w{b') for all W G £>, then if {a, a'} C A are such that Y(a) = b 
and Y{a') = b' , 1 z(a) = 1 z{cl') for all Z G cr(Y), so that a and a' belong to the same atom 
of ( A,a(Y )) and consequently b = b'. 

The rest of the claims are trivial. □ 

Now, a key result in this section will establish that, for a process X and a stopping time S 
thereof, cr(X s ) = , i.e. that the initial structure (with respect to S® T ) of the stopped process 

coincides with the filtration of the process at the stopping time - at least under suitable conditions. 

Our second lemma tells us that elements of J-g are, possibly non-measurable, functions of the 
stopped process X s . 
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Lemma 7.3 (Key lemma). Let X be a process (on II, with time domain T and values in E), S an 
F x -stopping time, A £ F x . Then the following holds for every {u;,u/} C f 1: If X t (uj) = X t (oj') 
for all t £ T with t < S(u) A S(u'), then 

S(oj) = X s (u) = X s (u/) and 1a(w) = 1 a(c*/)- 

Remark 7.4. This implies that if A £ F x , then 1 a = Fo X s for some function F : IinX’ -a- {0,1}. 
Thus, if X s : O -A Im7T s happens to be F x /‘2} mX -measurable (this would typically be the case if 
the range of X s is denumerable, and the sample paths of X are sufficiently regular), then it follows 
at once that cr(X s ') = F x . 

Proof. Define t l = f S(oj) A S(u'). If t = oo, then clearly S(u) = S( u/). If not, then {S < t] £ F x , 
so that there is a U & £* < s>[°> t ] with {S < t} = X\ 7*,(17). Then at least one of vj and uJ must belong 
to {5 < t}, hence to XX 1 XU). Consequently, since by assumption X|[ 0it ](w) = -X'|[o,t]0*; , )> both 
do. It follows that S(u>) = S(u'). In particular, X s ( lo) = X s (a/). 

Similarly, since A £ F x , An {5 < t} £ F x , so that there is a 17 £ (respectively U £ £® T ), 

with A n {S < t} = X\z' (U) (respectively A n {S < t} = X~ l (U)), when t < oo (respectively 
t = oo). Then IaH = lAn{S<t}M = lt/(^|[o,t]M) = M^lto,*]^')) = 1 An{S<t}(^ / ) = Ia(w') 
(respectively 1 a(w) = lAn{S<t}M = 1[/(^M) = 1 u(X(uj')) = lAn{S<t}(^') = 1 a(w'))- □ 

Our second lemma deals with the discrete case. 

Lemma 7.5 (Stopping times I). Let T = No and X be a process (on Q, with time domain No and 
values in E). For a time S : D —> No U {oo} the following are equivalent: 

(1) S is an F x -stopping time. 

(2) S is an F xS -stopping time. 

Proof. Suppose S is an F x ~ -stopping time. Let n £ No- Then for each m £ [0, n], {5 < m} £ F xS , 
so there is an E m £ with {S < m} = (77 s |[ 0im ]) _1 (K m ,). Then {5 = m} C X\^ m ](E m ) C 

{S < m}. Consequently {S < n} = U me[0tn] X\~^ m] (E m ) £ F x . 

Conversely, suppose S is an .F-^-stopping time. Let n £ No- For each m £ [0, n], {S' < rn} £ F x , 
hence there is an E m £ £®[°> rn l with {S < m} = X\ 7 1 , (F m ). Then {5 = m} C (77 5 |[ 0jm ]) _1 (K m ,) C 
{S < m}. Consequently {S < n} = U me[0)n ](X s |[o im ]) _1 (.E m ) £ F xS . □ 

The next step establishes that members of F x are, in fact, measurable functions of the stopped 
process X s - at least under certain conditions (and always in the discrete case). 

Proposition 7.6. Let X be a process (on LI, with time domain T and values in E), S an F x - 
stopping time. If any one of the three conditions below is fulfilled, then F x C cr(X s ) (where X s is 
viewed as assuming values in (E T , £® T )). 

(Ml) T = N 0 . 

(M2) InmY s C ImX. 
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(M3) (a) (Gl,G) is Blackwell for some a-field Q D F x V a(X s ). 

(b) a{X s ) is separable (in particular, this obtains if (ImX s ,£® T \ lmX s) is separable). 

(c) X s is constant on the atoms of o(X s ), i.e. (ImJ g , £® r | Tr ,-, y s) is Hausdorjf. 


Remark 7.7. 

(1) Condition |(M2)| is clearly quite a strong assumption, but will typically be met when X is 
the coordinate process on a canonical space. It is slightly weaker than condition (1.11) of 
(15. Paragraph 1.1.3] (see [15, Paragraph 1.1.3, p. 10, Theorem 6]). 

(2) Condition ( (M3)b| > is satisfied if there isaDc E T , with ImA 5 C T> , such that the trace 
u-field £® t \x> is separable. For example (when T = [0, oo)) this is the case if I? is a second 
countable (e.g. separable and metrizable) topological space endowed with its (consequently 
separable) Borel u-field, and the sample paths of X s are either all left- or all right-continuous 
(take V to be all the left- or all the right-continuous paths from I?! 0 ’ 00 )). 


(3) Finally, condition ((M3)c) holds if ( E ,£) is Hausdorff and so, in particular, when the 
singletons of E belong to £. 


Proof. Assume first (Ml) Let A £ Eg and n £ No- Then ifl{S = n}e F x , so A n {S = n} = 
(A'|[o y , n \)~ l {Z) for some Z £ But then A n {S = n} = (X s \\^ n ()~ l {Z) n {5 = n}. Thanks 

{S = n} £ a (X s ). A similar argument deals with the case where n = oo. 


to Lemma 


7.5 


Assume now 


(M2) Let A £ Fg . Then 1a = F o X for some f ® T /2^ 0,1 J'-measurable mapping 


F. Since ImA' 5 C ImA, for any u £ Q, there is an uj' £ 12 with A(a/) = X s (oj), and then thanks 
X s (u/) = X s (u), moreover, F o X s (u>) = F o A(a /) = 1 a ( w ') = 1a(w). It follows 


7.3 


to Lemma 
that 1a = F o X s . 

Finally assume [(M3)| We apply Blackwell’s Theorem. Specifically, thanks to ( |(M3)a ) (£l,G) 
is a Blackwell space and Fg is a sub-u-field of G] ((M3)a) and ((M3)b) imply that u(A‘ s ) is a 
separable sub-u-field of G■ Finally, thanks to ( |(M3)c ) and Lemma 7.3, every atom (equivalently, 
every element) of F(f is a union of atoms of (j(A’ s ). It follows that Fg C u(A 5 '). □ 


The continuous-time analogue of Lemma 7.5 is as follows: 


Proposition 7.8 (Stopping times II). Let T = [0, oo) and X be a process (on f l, with time domain 
[0, oo) and values in E), S : £1 —> [0, oo] a time. Suppose: 

(51) cr(A|[ 0t ]) and a(X SAt ) are separable, (InuA|[ 0 y],£®^) and (ImA' gAt ,£® T l TTT1 y SAt) Haus¬ 
dorff for each t £ [0, oo). 

(52) X s and X are both measurable with respect to a Blackwell a-field G on Q. 

Then the following are equivalent: 

(a) S is an F x -stopping time. 

(b) S is an F xb -stopping time. 
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Proof. Suppose first (b). Let t G [0, oo). Then {S < t} G F x . But F x = cr(X s |[ 0)t ]) C 
<t(^l | [o^]) = F X ■ This follows from the fact that every atom of cr(J^ 5 1 [ 0 ,t]) is a union of atoms of 
cr(X|[ 0 t ]) (whence one can apply Blackwell’s Theorem). To see this, note that if ui and c J belong 
to the same atom of cr(A|j 0) t]), then X|[o,t](w) = [ 0t ](co') (since (ImA|r 0) f],£®[ 0 ’^) is Hausdorff). 

But then X s (u) = X s (uj 1 ) for all s G [0, (S(u) At) A (S(u>') A t)], and so by Lemma |L3| (applied to 
the process X s and the stopping time S At of F xS ), (X s ) SAt (u ;) = ( X s ) SAt (vj'), i.e. -Y‘ 5 ’|[ 0i t](u;) = 
A 5 | [0;t] (u/). We conclude that u and belong to the same atom of <7(A 5 |[ 0 t t])- 

Conversely, assume (a). Let t G [0,oo). Then {S < t} G P X At , S At is an J rA -stopping time and 
thanks to Proposition 7.6, J~ X At C a(X SAt ) = o'(A'‘ s, |[o i t]). 


□ 


We now give the main result of this subsection. As mentioned in the Introduction, it generalizes 
canonical space results already available in the literature. 

Theorem 7.9 (A generalized Galmarino’s test). Let X be a process (on fl, with time domain T 
and values in E), S an T x -stopping time. 

(1) If T = N 0; then a(X s ) = F x . 

(2) Moreover, if X s is F x /E® T -measurable (in particular, if it is adapted to the stopped filtra¬ 
tion {F x AS )t£T) and either one of the conditions: 

(Gl) Trn X s C Tm X . 

(G2) (a) (f l,Q) is Blackwell for some a-field Q D F x . 

(b) a (X s ) is separable. 

(c) (ImA 5 ,£'® T | Im j s: s) is Hausdorff. 

is met, then the following statements are equivalent: 

(i) AeP x . 

(ii) 1a is constant on every set on which X s is constant and A G F x . 

(Hi) A G cr(A 5 ). 


Proof. The first claim, which assumes T = No, follows from Proposition 7.6 and the fact that 
automatically X s is F x /£® T -measurable in this case. 


In general, implication (i) => (ii) follows from Lemma 7.3 
follows. 


Implication (ii) => (iii) proceeds as 


Suppose first (Gl), Let 1 a be constant on every set on which X s is constant, A G F x . Then 


1a = Fo X for some £ 0T /2{ o,1 J'-measurable mapping F. Next, from ImA s C ImA, for any uj G Ii, 


there is an w'Gf1 with A (a/) = X s (u), and then thanks to Lemma 7.3, X s (u /) = X s (uj), so that 


by assumption, 1 a(w) = 1 a(w'), also. Moreover, FoX s (uj) = F o X(u>') = 1 a(w') = 1a(w). It 
follows that 1a = F o X s . 

Assume now|(G2)| Again apply Blackwell’s Theorem. Specifically, on account of (G2]|(a) (Ll,Q) 


is a Blackwell space and F x is a sub-cr-field of Q\ on account of (G2' b) and cr(X s ) C F x , a(X s ) 
is a separable sub-cr-field of Q. Finally, if is constant on every set on which X s is constant and 
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A E F x , then 1 a is a ^-measurable function (by (G2' 'a)), constant on every atom of a(X s ) (by 


(G2^ c)). It follows that 1 a is o-(A 5 )-measurable. 


The implication (iii) => (i) is just one of the assumptions. 


□ 


As for our original motivation into this investigation, we obtain: 


Corollary 7.10 (Observational consistency). Let X and Y be two processes (on 0, with time 
domain T and values in E), S an J~ x and an F'' -stopping time. Suppose furthermore X s = Y 5 . 
If any one of the conditions 
(01) T = N 0 . 

(02) ImA = ImY. 


or 


(03) (a) (respectively {0,%)) is Blackwell for some a-field Q D F x (respectively H D 

T y ) 

(b) a{X s ) (respectively a(Y s )) is separable and contained in F x (respectively J 7 ])). 

(c) (ImA 5 ,£® t | Trn ) and (ImY 5 ,£® T | Im ys) are Hausdorff. 
is met, then J~ x = Fg . 

Remark 7.11. If T = No, then rather than requiring that S' be a stopping time of both F x and 
J-' 1 , it follows from Lemma | t. 5| that it is sufficient for S' to be a stopping time of just one of them. 
The same is true when |(03)| obtains, as long as the conditions of Proposition 7.8 are met for the 
time S and both the processes X and Y. 


Proof. If | (01) | or |(03)| hold, then the claim follows immediately from Theorem |7.9 
If 


(02) holds, let A E Xg- , t E T. Then lAn{S<t} = F ° AT | [ 0 ,*] f° r some S‘ s> t 0 ’^/2^ 0,1 ^-measurable 


F. Moreover, for each w E there is an u/ E 0 with X{u') = Y(w), hence X(u) agrees with 


Y (w) = X(uj') on T 0 [0, S(w)], and thus thanks to Lemma 7.3, S(u) = S(u') and 1 ^(w) = 1 a(w'). 
This implies that F o Y|[ 0jt ](ct;) = F oI| [W ](w') = l An{s < t }(u/) = lAn{S<t}(^), i.e. lAn{S<q = 
F o Y|r 0 )t i. The case with t = oo is similar. □ 

We also have: 


Proposition 7.12 (Monotonicity of information). Let Z be a process (on Q, with time domain T 
and values in E) and U <V be two stopping times of T z . If either 
(Nl) T = N 0 
or else 
(N2) all of 

(a) (f2,£7) is Blackwell for some a-field Q D cr(Z v ) V a(Z u ). 

(b) (InaZ 1 , S® T \i mZ v) is Hausdorff. 
and 

(c) a(Z v ) is separable, 
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then a(Z u ) C a(Z v ). 


v\ 


Proof. In the discrete case the result follows at once from Theorem 7.9 In the continuous case, 
we claim that the assumptions imply that every atom of a(Z u ) is a union of the atoms of a(Z v ). 
To see this, let u and a/ belong to the same atom of er(Z' ); then, since (ImZ v ,S® T \ lmZV ) is 
Hausdorff, Z x (ca) = Z 1 (cj'), hence Lemma 7.3 implies that V(u) = V(uj') and U{u) = U{u'), and 
so a fortiori Z u (u>) = Z u (u /), which implies that u> and u/ belong to the same atom of a(Z u ). 
Now apply Blackwell’s Theorem. □ 


7.3. The completed case. We have studied in the previous subsection natural filtrations, now we 
turn our attention to their completions. For a filtration Q on fl and a complete probability measure 
P, whose domain includes Goo, we denote by Q P the completed filtration given by Gt c = Gt VA7, 
A f being the P-null sets; likewise if the domain of P includes a u-field A on II, then A P A V AT 
(= {A 1 from the domain of P such that 3A £ A with P(AAA/) = 0}). For any other unexplained 
notation, that we shall use, we refer the reader to the beginning of section 7.2 Recall in particular 
that T = No or T = [0, oo), Li is a set and (E,£) is a measurable space. 

First, all is well in the discrete case: 


Lemma 7.13. Let T = No and G be a filtration on Li. Furthermore, let P be a complete probability 

—P 

measure on LI, whose domain includes Goo and let S be a G -stopping time. Then: 

(1) S is P-a.s. equal to a stopping time S' ofG; 

_p _p 

(2) for any G-stopping time U which is P-a.s. equal to S, Gu = Qs> 
and 

_p 

(3) if U is any random time, P-a.s equal to S, then it is, in fact, a G -stopping time, and 



Proof. 

(1) For each n £ No, we may find an A n £ G n , such that {5 = n} = A n , P-a.s. Then 
S' = f EneNo nl A n + oo 1 D\u meNQ A m is a ^-stopping time, P-a.s. equal to S. 

_p _p 

(2) Now let U be any ^-stopping time, P-a.s. equal to S. To show Gu C £/g, it suffices to 

_p _p _p 

note that (i) A f is contained in G$ an d (ii) Gu C G$- Conversely, if A £ t/g, then for 
each n £ No U {oo}, A n {S' = n} = B n , P-a.s., for some B n £ Gn, and hence the event 
CneN 0 u{oo} E n n {17 = n} belongs to Gu, an d is P-a.s. equal to A. 

(3) Finally, let U be a random time, P-a.s equal to S. For each n £ No U {oo}, there is 

_p 

then a C n £ G n with {U = n} = {S = n} = C n , P-a.s., so U is a G -stopping time. It 

follows that we can find U', a t/-stopping time, P-a.s. equal to [7, hence S, and thus with 
—p -p —p 

G s = Gu> =Qu- 
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Corollary 7.14. Suppose that T = No; X and Y are processes (on 1 1, with time domain No and 
values in E); P A and are complete probability measures on 17 whose domains contain X x and 

__ pX _ _pY 

X^, respectively, and with the same null sets. Suppose furthermore S is an X x and an X Y 

_pX _pX _pV _ 

stopping time, with X s = Y s , P A and P^ -a.s. Then X x s = a(X s ) = o/y 5 ) = X Y s . 


Proof. From Lemma 7.13 we can find stopping times U and V of X x and X' 1 , respectively, both P A 
and P* -a .s. e qual to 5. T _ 

again, that X x s = X\j 

□ 


Theorem 

-rP^ 


7.9 


and Lemma 


re event {X u = Y v } is P A and P* -almost certain. It then follows, from 

p.V _pX _ pX _pY _pY 

- T x = a{X u ) = (j{Y v ) = X}' ~ 


7.13 


v 


X Y S , as desired. 


Corollary 7.15. Let T = No, X be a process (on 12, with time domain No and values in E), P be 
a complete probability measure on 12 whose domain contains X x V X xS and S : 12 -A T U {oo} be 
a random time. Then the following are equivalent: 

-P 

(1) S is an X x -stopping time. 

——-p 

(2) S is an E x -stopping time. 


Proof. That (1)=>(2) is clear from Lemma 7.13, Lemma 7.5 and the fact that two processes, which 
are versions of each other, generate the same filtration, up to null sets. For the reverse implication, 
one can proceed as in the relevant part of the proof of Lemma 7.5, adding P-a.s. qualifiers as 
appropriate. □ 


The continuous-time case is much more involved. Indeed, we have the following significant 
negative results: 


Example 7.16. Consider the following setup. 12 = (0, oo) x {0,1}/ T is the product of the Lebesgue 
a-field on (0, oo) and of the power set on {0,1}; P = Exp(l) x Unif({0,1}) is the product law on X 
(which is complete); e (respectively I) is the projection onto the first (respectively second) coordinate. 
The process N is given by N t = (2 —e) 1 r 0 ,t] (e)7 for t E [0, oo) (starting at zero, the process N departs 

from zero at time e with unit positive drift, or remains at zero for all times, with equal probability, 

-p 

independently of e). Its completed natural filtration, X N , is already right-continuous. 

For, if t E [0, oo), X N t+ = X x + ; so let A E X(^_, we show A E X x . (i) A n {e = t} is 
P -negligible, (ii) Clearly A = N^ 1 {G), for some measurable G C Ml 0 ’ 00 ). Then define for each 
natural n> 1/t (when t > 0), L n : Ml 0,t l —> Ml 0 ’ 00 ), by demanding 


L n (u)(u) 


co(u ), for u <t 

u(t) + (u- , for u > t 


(u E [0, oo), uj E Ml 0 ’*!), a measurable mapping. It follows that for t > 0, for each natural n > 1/t, 
iV - 1 (G) n {e < t — 1/n} = A^|^ 1 t j(L“ 1 (G)) n {e < t — 1/n) E X x . (in) For each natural n, 
An {e > t + 1/n} = AI|F 1 t+ i/ n ](G n ) n {e > t + 1/n} for some measurable G n C Ml 0 ’ t+1//ri l, so 
A n {e > t + 1/n} is 0 or {e > t + 1/n} according as 0 is an element of G n or not (note this 
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is a “monotone'’ condition, in the sense that as soon as we once get a non-empty set for some 
natural n, we subsequently get {e > t + 1/m} for all natural m> n). It follows that A n {e > t} = 
U n£ N(^4 n {e > t + 1/n}) E {0, {e > i}} C F x . 

Further, let U be the first entrance time of the process N to (0,oo). By the Debut Theorem, this 
is a stopping time of F N , but it is not P-a.s. equal to any stopping time of F N at all. 

For suppose that it were P-a.s. equal to a stopping time V of F N . Then there would be a set 
if, belonging to F, of full P -measure, and such that V = U on If. Tracing everything (F, P, N, 
a, e, V) onto If, we would obtain (F’, P', N', a', e' , V'), with (i) V' equal to the first entrance 
time of N' to (0, 00 ) and (ii) V' a stopping time of F N ', the natural filtration of N'. Note that 
Nj. = a'(t — e / )l[o ) t](e / ), t E [0, 00 ). Now take C fi 7 with a{uS) = 1, a(u/) = 0, denote 

t = e(u). Then N '| [ 0)t ] (cj) = JV'I^ so uj and u' should belong to the same atom of F x '; yet 

{V' < t} E F x , with Ijy = 1 and 1 {v 7 <*}(<*/) = 0 , a contradiction. 

_p _p clof 

Moreover, F N jj / a{N u ) , since the event A = {U < 00 } = {a = 1} that N ever assumes a 

-p - P _._ p 

positive drift belongs to F N v (which fact is clear), but not to a(N u ) = cr(0) , the trivial a-field 
(it is also obvious; P(a = 1) = 1/2 ^ {0,1}/. o 

Example 7.17. It is even worse. Let Ll = (0, 00 ) X {—2,—1,0} be endowed with the law P = 
Exp(l) x Unif({—2, — 1,0}), defined on the tensor product of the Lebesgue a-field on (0,oo) and 
the power set of {— 2, —1,0}. Denote by e, respectively I, the projection onto the first, respectively 

def 

second, coordinate. Define the process X by Xt = (t — e)l[ 0 i t](e)I, t E [0,oo), and the process 

def 

Y t = (—l)(t — e)l[o j i](e)l/_i j _ 2 } o I, t E [0,oo). The completed natural filtrations of X and Y are 
already right-continuous. The first entrance time S of X into (— 00 ,0) is equal to the first entrance 

_p _p 

time of Y into (— 00 , 0), and this is a stopping time of F x as it is of F'* (but not of F x and 
not of F y ). Moreover, X s = 0 = Y' s . 

Consider now the event A = {/ = — 1}. It is clear that A E F x 5 . However, A (f F * s . For, 
assuming the converse, we should have, P-a.s., l J 4 n{S<i} = ^ ? °^l[o,i] f or some, measurable, F. In 
particular, since A n {S' < 1} has positive probability, there should be an uj E A n {5 < 1} with 
F(Y\ m (u)) = 1. But also the event {I = —2} n {S < 1} has positive probability and is disjoint 
from A n {5 < 1}, so there should be an u / E {/ = —2} n {S’ < 1} having F(y|[ 01 ](t</)) = 0. A 
contradiction, since nevertheless Y|[ 0 i i](w') = Y|[ 0 j i](cj). o 


The problem here is that in completing the natural filtration the apparently innocuous operation 
of adding all the events negligible under P is done uncountably many times (once for every deter¬ 
ministic time). In particular, this does not correspond to a single completion of the sigma-field 
generated by the stopped process: completions are not always harmless. 


Furthermore, it is not clear to us what a sensible direct ‘probabilistic’ analogue of Lemma 7.3 
should be, never mind how to go about proving one. 

However, the situation is not entirely bleak, since positive results can be obtained at least for 
foretellable/pr edict able stopping times of quasi-left-continuous filtrations - as in the case of discrete 
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time - by an indirect method; reducing the ‘probabilistic’ to the ‘measure-theoretic’ case. We use 
here the terminology of [3, pp. 127 and 137, Definitions IV.69, IV.70 and IV.84], i.e. given a 
filtration G and a probability measure Q on Gl, whose domain includes Goo'. 

Definition 7.18. a random time S : Gl — > [0, oo] is predictable relative to G if the stochastic in¬ 
terval [T, oo[ is predictable. It is Q-foretellable relative to G if there exists a Q-a.s. nondecreasing 
sequence (S n ) ra >i of £/-stopping times with S n < S, Q-a.s for all n > 1 and such that, again Q-a.s., 

lim S n = S, S n < S for all n on {5 > 0}; 

n—>oo 

foretellable, if the a.s. qualifications can be omitted. Finally, G is quasi-left-continuous if 
Gt = Gt~ for all predictable times T of Q. 


Note that (i) the property of predictability is invariant under passage to the right-continuous 
augmentation of a filtration, and (ii) in a P-complete filtration (P itself assumed complete), the 
notions of predictable, foretellable and P-foretellable stopping times coincide [3j p. 127, IV.70; p. 
128, Theorem IV.71 and p. 132, Theorem IV.77]. 

The following is now a complement to [31 p. 120, Theorem IV.59 and p. 133, Theorem IV.78] [9j 


p. 5, Lemma 1.19], and an analogue of the discrete statement of Lemma 7.13 


Proposition 7.19. LetT = [0, oo), G be a filtration on Gl. Let P be a complete probability measure 


—P 

on fl, whose domain includes Goo and S be a predictable stopping time relative to G that we assume 
is quasi-left-continuous. Then: 

(FI) S is P -a.s. equal to a predictable stopping time W of Q. 

_p _p 

(F2) Moreover, if U is any G-stopping time. P -a.s. equal to S, then G s = Gu ■ 

_p 

(F3) Finally, if S' is another random time, P -a.s equal to S, then it is a predictable Q -stopping 
—p —p 

time, and Gs = Gs> ■ 

Proof. 

(FI) This is contained in [31 p. 133, Theorem IV.78]. 

_p _p 

(F2) Now let U be any C/-stopping time, P-a.s. equal to S. The inclusion G S T) Gu is obvious. 

_ p _p _p 

Then take A £ Q s . Since A £ Goo = Goo , there is an A’ £ Goo, such that A' = A, P-a.s. 

—P 

Furthermore, thanks to quasi-left-continuity, Sa = -S'1a + oo1^\a is again Q -predictable 
m p. 418, Proposition 22.15]. Hence, by |(F1)[ there exists V , a £/-stopping time, with 
V = Sa, P-a.s. So, 

A = (A 1 n {U = oo}) U {V = U < oo} £ Gu, P-a.s. 

_p 

(F3) Finally let S' be a random time, P-a.s. equal to S. Clearly, it is a predictable G -stopping 
time. Moreover, by |(F1)[ we can find W, a ^-stopping time, P-a.s. equal to S and hence 


(by hypothesis) S'. It follows from (F2) 


that Gs = Gw = Gs 1 ■ 


□ 
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From this we can obtain easily some useful counterparts to the findings of section 7.2 in the 
continuous case: Corollaries |7.20t |7.22| and |7.23| below. They can be applied to completions of 
Blackwell spaces in conjunction with (in this order) (i) the fact that a standard Borel space-valued 
random element measurable with respect to the completed domain of a probability measure Q is 
Q-a.s. equal to a random element measurable with respect to the uncompleted domain of Q (Q 
being the completion of Q) [TTI p. 13, Lemma 1.25] and (ii) part |(F 1 J{of Proposition 7.19 


Corollary 7.20. Let T = [0, oo), Z be a process (on 12, with time domain T and values in E), P 

_p 

be a complete probability measure on 12, whose domain includes F^, and W be an F z -predictable 

-P 

stopping time with F z quasi-left-continuous. If, for some process X P -indistinguishable from Z 

_p _p 

and a stopping time S of F x , with S = W, P-a.s., Condition (M3) holds, then F z w C a(Z w ) . 


Remark 7.21. The reverse inclusion a(Z w ) C F z w is usually trivial. 


Proof. According to Proposition 


7.6 


F% C ct{X s ). Also F x = F z and u(A s ) = a(Z w ) . 

to the stopping time W 


7.19 


Taking completions in Fif C cr(A' b ), by applying (F2) of Proposition 

_ p - 

of F x (which is P-a.s. equal to the stopping time S of F x ), we obtain: 


F Z W =F X W = F X Ca{X s ) = a{Z w ) , 


as desired. 


□ 


Corollary 7.22. Let T = [0, oo); let Z and W be two processes (on 12, with time domain [0, oo) 
and values in E); P z and P" be probability measures on 12, with the same null sets, and whose 

_ pZ _p W 

domains include and F^, respectively; V a predictable F z - and F w -stopping time, with 

_pZ _p W 

F z and F w quasi-left-continuous. Suppose furthermore Z'' = W v , P z and P" -a.s. 

If there exist two processes X and Y, indistinguishable from Z and W, respectively, and stopping 
times S and U of F x and F 1 , respectively, with S = U = V, P z and P u -a.s. and such that 


the pairs (X,S) and ( Y,U ) each satisfy Condition (M3), then F z v = a(Z' i ) = a(W v ) = 

_p w 

F w v ■ 


Proof. The claim follows from Corollary 

_pZ _pZ 


7.20 


a(ZV) 


and the fact that again <r(A‘ 9 ) C F x implies 
C F Z y ; with a comparable statement holding for W. □ 


Corollary 7.23. Let T = [0, oo); X be a process (on 12, with time domain [0,oo) and values in 
E); P a complete probability measure on 12, whose domain includes F x ; S and W two predictable 

-rP -rP 

stopping times of F x with S < W and F x quasi-left-continuous. Let U and V be two stopping 
times of the natural filtration of a process Z, P -indistinguishable from X, P-a.s. equal to S and W, 
respectively, with U < V, and such that: 

(Ql) (12,17) is Blackwell for some a-field Q D cr(Z v ) V a(Z u ), 

(Q2) (ImZ v , S® T \i mZ v) is Hausdorff 
and 
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(Q3) a(Z v ) is separable. 
Then a(X s f C a(X w ) P . 


Remark 7.24. Proposition 7.1£ FI) establishes the existence of U and V f , stopping times of J- z , P- 
a.s. equal to S and W. respectively, with U < V\ P-a.s. Defining V = f V't(U < V') + U1(U > V'), 
V is also a stopping time of T 7, , P-a.s. equal to W, and it satisfies U < V with certainty. The 


question is whether these stopping times satisfy conditions (Q1)-(Q3) 


Proof. We see that a(X s ) = a(Z u ) and a(X w ) = a(Z v ) . 
the result. 


Applying Proposition 


7.12 


gives 

□ 


lary 


We are not able to prov ide a useful counterpart to Proposition 7.8 In the notation of Corol- 

7 p 


7.23 


Proposition 


7.19 


does say that, given a predictable stopping time P of T xP , there is a 
predictable stopping time U of T zP , P-a.s. equal to P. But this does not a priori say that U is a 
stopping time of J - Z ' J , so one cannot directly apply Proposition 


7.8 
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Appendix A. Miscellaneous technical results 

Throughout this appendix (fi, T, P) is a probability space; E denotes expectation with respect 
to P. 

Lemma A.l (On conditioning). Let X : PI — > [— 00 , 00 ] be a random variable, and Qi C T, 
i = 1,2, be two sub-a-fields of T agreeing when traced on A G C/i O t/ 2 - Then, P-a.s. on A, 
E[X|C/i] = E[A"|C? 2 ] ; whenever X has a P-integrable positive or negative part. 

Proof. IaZ is ^-measurable, for any Z C/i-measurable, by an approximation argument. Then, 
P-a.s., 1 ^ 4 E[AT|] = Efl^Al^]) by the very definition of conditional expectation. □ 

Recall Definition 14.51 

Lemma A.2. Suppose that PL is a (respectively a P-complete) filtration on PL in discrete (T = No,) 
or continuous (T = [0,oo)J time and that a sequence ( S n ) ne ^ of its stopping times accesses infinity 
pointwise (respectively P-a.s.) on A C PL. Then PLoo\a = V ne R^S n U- 

Remark A.3. Note that for any £ C 2 s * and icSl, cjq{C)\a = cta{T\a)i so there is no ambiguity 
in writing V ne nPls n \A- 

Proof. The inclusion PLoo\a A Vng^'Rs'J^ is clear. To establish the reverse inclusion let t G T and 
B G Pi t . Then (respectively P-a.s.) BnA = U“ =1 (Bfl{5„ >l})fld with B O [S n > f} e PLs n ■ □ 

The following lemma uses the notation, and is to be understood in the context, of sections [3] 
and[4j 

Lemma A.4. Let {c,d} C C and A C PL. Suppose that (<S n ) ne n is a sequence in G accessing 
infinity {a.s.} on A for the controls c and d, and for which c ~s n d for each n G N. Then 
@oo\a = Qto\A- 

If further, A G Q^ c for all n G N, and (S%( w)) nS N is nondecreasing for {P h -almost} every tv G A, 
each h G {c, d}, then P c |gg 0 and P d | gd° agree when traced on A. 
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Remark A.5. We mean to address here abstractly the situation when the two controls c and d agree 
for all times on A. 


Proof. By stability under stopping, certainly P c \g c c agrees with P“Ud for each n G N, while 

«5>n Wed 

°n _ 

= Sf;) accesses inhnity {P c -a.s. and P^-a.s.} on A. Then apply Lemma |A.2| to obtain 
Q'oo\a = w(U neN ^cU) = crA (U n gif^,, | a ) = If, moreover A e ^ for all n € N, then 

the traces of P c and P rf on A agree on U ne pj^cU- Provided in addition (5^) ne ^ is {P c -a.s.} 
nondecreasing on A, the latter union is a 7r-system (as a nondecreasing union of er-fields, so even an 
algebra) on A. This, coupled with the fact that probability measures which agree on a generating 
7r-system are equal, yields the second claim. □ 


Lemma A. 6 (Generalised conditional Fatou and Beppo-Levi lemmas). Let Q C T be a sub-a-field 
and (f n ) n >i cl sequence of [— 00,00 \-valued random elements, whose negative parts are dominated 
P-a.s by a single P -integrable random variable. Then, P-a.s., 

E[liminf f n \Q] < lirninf E[f n \Q]. 

n—>00 n—> 00 

If, moreover, (f n )n> 1 is P-a.s. nondecreasing, then, P-a.s., 

E[ lim f n \G] = lim E[f n \G]. 

n—¥ 00 n—>00 

Proof. Just apply the conditional version of Fatou’s Lemma (respectively of the Beppo-Levi Lemma) 
to the P-a.s. nonnegative (respectively nonnegative nondecreasing) sequence f n +9 where g is the P- 
integrable random variable which P-a.s. dominates the negative parts of the f n . Then use linearity 
and subtract the P-a.s. finite quantity E[g|<J]. □ 

The following is a slight generalization of m Theorem A2], 

Lemma A.7 (Essential supremum and the upwards lattice property). Let G C P be a sub-a-field 
and X = (X\)x£j\ a collection of [— 00,00 \-valued random variables with integrable negative parts. 
Assume furthermore that for each {e, M} C (0,oo), X has the “(e, M)-upwards lattice property”, 
i.e. for all {A, A'} C A, one can find a A" G A with Xy > (M A A^) V (M A Xy) — e P-a.s. Then, 
P-a.s., 

E[P-esssup AeA X A |£] = P -esssup AgA E [X\ \Q], (A.l) 

where on the right-hand side the essential supremum may of course equally well be taken with respect 
to the measure P\g. 

Proof. It is assumed without loss of generality that A 7 ^ 0, whence remark that P-esssup AgA X A has 
an integrable negative part. Then the inequality 

E[P-esssup AgA X A | Q] > P-esssup AgA E[A' A |£] 


is immediate. 
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Conversely, we show first that it is sufficient to establish the reverse inequality for each truncated 
(X\ A N )\ gA family, as N runs over N. Indeed, suppose we have P-a.s. 

E[P-esssup AgA X A A N\Q] < P-esssup AgA E[A' A A N\Q] 

for all IVeN. Then a fortiori P-a.s. for all N € N, 


E[P-esssup AgA X A A N\Q] < P-esssup AgA E[X A |£] 


and generalised conditional monotone convergence (Lemma A. 6 ) allows to pass to the limit: 


E[ lim P-esssup AgA A A A N\Q] < P-esssup AgA E[A A |C/] 

N^-oo 

P-a.s. But clearly, P-a.s., lini/v-^oo P-esssup AgA A A A N > P-esssup AgA A A , since for all A £ A, we 
have, P-a.s., X\ < lim/v^oo X\ A N < liniTv^oc P-esssup /igA A A1 A N. 


Thus it will indeed be sufficient to establish the “<-inequality” in (A.l) for the truncated families 


and so it is assumed without loss of generality (take M = N ) that X enjoys, for each e 6 (0, oo), 
the “e-upwards lattice property”: for all {A, A'} C A, one can find a A" 6 A with Xy> > X\\/Xy — e 
P-a.s. 

Then take (A n ) n >i C A such that, P-a.s., P-esssup AgA X A = sup n>1 X\ n and fix <5 > 0. Recur¬ 
sively define (A(J n >i C A so that, X A / = A Al while for n E N, P-a.s., Xy i > Xy n V X\ n+1 — 5/2 n . 
Prove by induction that P-a.s. for all n G N, Xy n > maxi<fc< n (AT Afe — Ya=i V2 ; )> so that 
lim infn._ 5 . 0 c Xy n > sup ngN X\ n — 5 , P-a.s. Note next that the negative parts of (A A ;J ng f>} are 
dominated P-a.s. by a single P-integrable random variable. By the generalised conditional Fatou’s 
lemma (Lemma A. 6 ) we therefore obtain, P-a.s., P-esssup AgA E[A" A |t?] > liminf^^oo E[X A ^ \G] > 
Ejliminfn-^oo Xy \Q] > E[P-esssup AgA X A |^] — 5. Finally, let 5 descend to 0 (over some sequence 
descending to 0). □ 


Lemma A.8. 

(i) Let A and B be two cr-fields on LI, F C fl, V : fl —>■ [— 00 , 00 ], such that A\f = B\f with 
F e AnB. Then Vt f is B-measurable ifV is A-measurable. 

(ii) Let A and B be two filtrations in continuous (T = [0,oo)^ or discrete (T = Noj time on LI, 
V be a real-valued process on f l with time domain T, P be a map P : Q ^ T U { 00 }, and 
A C Ll. Assume that for all t 6 T, 

(I) A\{t<p} = Bt\{t<p}> A\{t>p}rA = &t\{t>p}rA 

and 

(II) the events {t < P}, An {P < t} belong to Bt n At- 

Then Ll[ 0 i p] and R1(|p ) 0 O |)1a are B-adapted (respectively, predictable) if V is A-adapted 
(respectively, predictable). 


Proof. The first part is clear. The second part in discrete time follows at once. In continuous time, 
adaptedness also follows at once from the first part. For predictability, one notes that the class of 
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processes V for which Vl[o,p] and V1 (|p i00 d1a are £>-predictable contains the multiplicative class 
of all left-continuous M-adapted processes. The Functional Monotone Class Theorem allows us to 
extend this claim to all M-predictable processes. □ 
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